Medical support system and method for supporting medicine

Abstract

To provide a mechanism which can improve an operational technique for a living body.SOLUTION: A medical support system includes: an acquisition unit for acquiring medical image data of a living body; a first generation unit for generating base data of 3D data related to a tissue specified on the basis of a detection amount characteristic from the medical image data: a second generation unit for generating characteristic part data as 3D data related to a characteristic part specified on the basis of the image processing of the base data; and an output unit for outputting the base data and the characteristic part data as operation assisting data.SELECTED DRAWING: Figure 21

Description

[Technical field]

[0001] The present technology relates to a medical support system and a medical support method. [Background technology]

[0002] As a background technology of this technical field, there is JP 2023-002280 A (Patent Document 1). This publication states that "a) a function in which a first dentist performs user registration from a first user terminal 111 via a network, a function in which the first dentist uses a 3D model or the like to explain a dental case, transmitted from the first user terminal, receives via a network a video or the like of the explanation of the dental case, a function in which the video or the like received by the function in (b) is classified and displayed on the second user terminal 112 of the first viewer or the first user terminal via a network (c), and when the first viewer selects a video or the like classified and displayed on the second user terminal, a function in which the selected video or the like is displayed on the second user terminal via a network (d), and a function in which the first viewer performs a procedure for purchasing a 3D model or the like from the second user terminal via a network (f) A dental comprehensive training site 118 having a program that realizes this function" (see abstract). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2023-002280 A Summary of the Invention [Problem to be solved by the invention]

[0004] Patent Document 1 discloses a business model for a comprehensive dental training site. However, it does not disclose a mechanism for improving treatment techniques for living organisms. Therefore, the present technology provides a mechanism that can improve treatment techniques for living organisms. [Means for solving the problem]

[0005] In order to solve the above problems, for example, the configurations described in the claims are adopted. The present application includes multiple means for solving the above-mentioned problems. As an example, the present application provides a medical support system including an acquisition unit that acquires medical image data of a living body, a first generation unit that generates base data, which is 3D data related to tissue identified based on detection quantity characteristics in the medical image data, a second generation unit that generates feature part data, which is 3D data related to a feature part identified based on image processing of the base data, and an output unit that outputs the base data and the feature part data as treatment support data. Effect of the Invention

[0006] According to the present technology, a mechanism can be provided that can improve treatment techniques for living organisms. Problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 shows an example of the overall configuration of a dental treatment support system 100. [Diagram 2] FIG. 2 shows an example of the hardware configuration of the management server 101. [Diagram 3] FIG. 3 shows an example of the hardware configuration of the user terminal 102. [Figure 4] FIG. 4 is an example of user information 400. [Diagram 5] FIG. 5 is an example of the AR model data 500. [Figure 6] FIG. 6 is an example of training data 600. [Figure 7] FIG. 7 shows an example of check data 700 in the case of implant placement. [Figure 8] FIG. 8 is an example of NG area data 800. [Figure 9]FIG. 9 shows an example of a treatment support data generation flow 900. [Figure 10] FIG. 10 is an example of a base data generation flow 1000. [Figure 11] FIG. 11 is an example of a feature portion generation flow 1100. [Figure 12] FIG. 12 is an example of a hollow portion identification flow 1200. [Figure 13] FIG. 13 is an example of a surface morphology feature identification flow 1300. [Figure 14] FIG. 14 shows an example of a treatment support data output flow 1400. [Figure 15] FIG. 15 is an example of a treatment evaluation process flow 1500. [Figure 16] FIG. 16 is an example of a CT image based on the CT data. [Figure 17] FIG. 17 is an example of an AR model. [Figure 18] FIG. 18 shows another example of an AR model. [Figure 19] FIG. 19 is another example of an AR model. [Figure 20] FIG. 20 is an example of an overlaid image of the AR model and NG area of ​​the mandible. [Figure 21] Figure 21 is an example of an image in which the AR model of the mandible and the NG area of ​​the mandibular canal are superimposed. [Figure 22] FIG. 22 is an example of an AR model image of the mandible. [Diagram 23] FIG. 23 shows an example of a state in which a guide is attached to a dental arch model. [Figure 24] FIG. 24 shows an example of a dental arch model. [Diagram 25] FIG. 25 shows an example of a support base for a dental arch model. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, a medical support system, a medical support method, a program, and a data structure according to the present technology will be described with reference to the drawings. Note that in each drawing, configurations having the same functions may be omitted from the description and reference numerals may be omitted.

[0009] [Dental treatment support system] FIG. 1 is an example of a configuration diagram of a dental treatment support system 100 (an example of a medical support system) according to an embodiment. The dental treatment support system 100 according to the present technology is configured to be able to evaluate treatment using a dentition model (an example of a biological model) on the spot during the training stage or treatment stage of dental treatment. In this embodiment, the present technology will be described using an example of a case in which implant insertion surgery is practiced using a patient's dentition model as a treatment in the dental field.

[0010] As shown in FIG. 1, the dental treatment support system 100 includes one or more management servers 101 and one or more user terminals 102. The dental treatment support system 100 may additionally include, for example, a 3D printer 103. Each of the one or more user terminals 102 is configured to be connectable to one or more management servers 101 via a network. The network may be wired or wireless, and each terminal is capable of transmitting and receiving information to and from each other via the network. The user terminal 102 may be provided separately from the management server 101, or may be provided integrally with the management server 101.

[0011] The management server 101 and the user terminal 102 in the dental treatment support system 100 (hereinafter, the management server 101 and the user terminal 102 may be simply referred to as "terminals") may be, for example, a mobile terminal such as a smartphone, tablet, mobile phone, or personal digital assistant (PDA), or may be a wearable terminal such as glasses (including goggles), wristwatch, or clothing. Each terminal may be a stationary or mobile computer, or a server located on the cloud or a network. From the viewpoint of functionality, each terminal may be a VR (Virtual Reality) terminal, an AR (Augmented Reality) terminal, or an MR (Mixed Reality) terminal. Alternatively, each terminal may be a combination of a plurality of these terminals. For example, a combination of one smartphone and one wearable terminal may function logically as one terminal. Each terminal may also be an information processing terminal other than these.

[0012] Each terminal and management server 101 of the dental treatment support system 100 may optionally include a processor that executes an operating system, an application, a program, etc., a main storage device such as a RAM (Random Access Memory), an auxiliary storage device such as an IC card, a hard disk drive, an SSD (Solid State Drive), a flash memory, etc., a communication control unit such as a network card, a wireless communication module, a mobile communication module, etc., an input device such as a touch panel, a keyboard, a mouse, a voice input device, a motion controller, an input device that detects motion by capturing an image of a camera unit, an input device such as a GPS, a gyro sensor, an acceleration sensor, etc., and an output device such as a monitor, a display, a printer, a voice output device, an oscillator, etc. The output device may be a device or terminal that transmits information to be output to an external monitor, a display, a printer, a device, etc.

[0013] The main memory stores various programs and applications (software modules), and the processor executes these programs and applications to realize each functional element of the overall system. Each module may be implemented as an independent program or application, or as a subprogram or function within a single integrated program or application. Each module may also be implemented as hardware by integrating circuits or employing a microcomputer (hardware module).

[0014] Furthermore, each module may be realized by a single processor or by multiple processors. Also, each module may be provided in a single terminal (including a management server) or may be provided separately in two or more terminals (including a management server) connected to each other via a network. Also, each module may be provided in each or any one or more of two or more terminals (including a management server) connected to each other via a network. Cases in which some modules are realized in a different country from the other modules are also considered.

[0015] In this specification, each module is described as an entity (subject) that performs processing, but in reality, processing is performed by a processor executing various programs, applications, and the like.

[0016] The auxiliary storage device stores various databases (DB). A "database" is a set of data that is organized and collected so as to be compatible with any data operation (e.g., extraction, addition, deletion, overwriting, etc.) from the processor or an external computer. The auxiliary storage device is a functional element (storage unit) that stores one or more sets of data. There are no limitations on the method of implementing the database, and it may be, for example, a database management system, spreadsheet software, or text files such as XML and JSON. Some or all of this information may be stored in a relational database or a non-relational database. The database may be provided independently in a state that is connectable to the processor, etc.

[0017] [Management Server] 2 illustrates an example of a hardware configuration of the management server 101. The management server 101 is an element that manages this dental treatment support system 100. The management server 101 is typically a terminal used by an administrator who manages and operates this dental treatment support system 100. The management server 101 is configured, for example, by a computer server or the like arranged on a cloud.

[0018] The management server 101 includes a main storage device 201 and an auxiliary storage device 202. The management server 101 also includes a processor 203, an input device 204, an output device 205, and a communication control unit 206 as described above.

[0019] The main memory device 201 stores programs and applications such as a user terminal management module 211, a dental CT acquisition module 212, an AR data generation module 213, a dentition model generation module 214, a NG area data generation module 215, a server-side evaluation module 216, and a treatment support data output module 217. Each functional element of the management server 101 is realized by the processor 203 executing these programs and applications stored in the main memory device 201. Note that some of these modules, such as the server-side evaluation module 216 and the treatment support data output module 217, are optional components of the management server 101 and may be omitted.

[0020] The auxiliary storage device 202 stores information necessary for the operation of the dental treatment support system 100. For example, the auxiliary storage device 202 stores user information 400, dental CT data 220, dentition model data 230, AR model data 500, training data 600, check data 700, and NG area data 800. Details of these pieces of information will be described later.

[0021] Each functional element of the management server 101 will now be briefly described. The user terminal management module 211 manages the operation of the user terminal 102. The user terminal management module 211 cooperates with a management module 311 of the user terminal 102 and the like to control basic operations for realizing the functions of the dental treatment support system 100 in the user terminal 102.

[0022] For example, the user terminal management module 211 cooperates with the management module 311 of the user terminal 102 to output (display) the homepage, login page, menu page of treatment contents, etc. of the dental treatment support site provided on the web by the management server 101 to the output device 305 such as a display of the user terminal 102. In addition, the user terminal management module 211 cooperates with various modules of the user terminal 102, for example, to acquire information input from the user terminal 102 via these pages, and output information to the user terminal 102. For example, the user terminal management module 211 can operate based on information (various information, instructions, etc.) acquired from the user terminal 102, and output (store) the acquired information to the auxiliary storage device 202 as user information 400, etc. In addition, information such as AR model data corresponding to the selected treatment contents can be output to the user terminal 102.

[0023] The dental CT acquisition module 212 acquires dental CT data. The dental CT data is an example of medical image data of a living body. The medical image data of a living body broadly includes information that can visualize the internal structure and function of a living body as an image for the diagnosis and treatment of a disease, and may be any one or a combination of two or more of data acquired by X-ray photography, computed tomography (CT), nuclear medicine diagnosis, magnetic resonance imaging (MRI), ultrasonic tomography, etc. These medical image data may be reconstructed so that the internal structure and function of a living body can be visualized as an image.

[0024] Furthermore, in this embodiment, an example is shown in which medical / dental CT (Computed Tomography) data of the stomatognathic region is used as dental CT data, but it is not limited to this as long as it can acquire three-dimensional morphological information of biological tissues of the stomatognathic region. The dental image may be, for example, one or a combination of two or more of various medical / dental image data such as plain / contrast CT data, magnetic resonance imaging data, digital X-ray imaging data, computed radiography data, single photon emission computed tomography imaging data, positron emission computed tomography imaging data, and ultrasound imaging data. Although not limited thereto, the dental CT data may be, for example, in accordance with the DICOM (Digital Imaging and Communications in Medicine) communication standard, and the data format may be HL7 (Health Level Seven). Note that the dental CT data referred to in this specification may include, but is not limited to, so-called dental cone beam CT optimized for CT imaging of teeth, jaws, and faces.

[0025] The dental CT acquisition module 212 may acquire dental CT data from a CT imaging modality installed in a dental clinic or the like, or may acquire imaging data that has been captured in advance and stored in a cloud database or various recording media. The dental CT acquisition module 212 stores the acquired dental CT data in the auxiliary storage device 202 as dental CT data 220.

[0026] The AR data generation module 213 includes a first generation module 213A and a second generation module 213B according to the present technology. The AR data generation module 213 generates Augmented Reality (AR) display data from dental CT data. In the present technology, the AR data is not particularly limited as long as it can display virtual visual information superimposed on a real-world image on a computer, and may be simple Augmented Reality data or Mixed Reality (MR) data. The AR data may also include audio information. The AR data generation module 213 typically generates data (3DCT data) corresponding to a three-dimensional (3D) model of the stomatognathic region from the CT data of the stomatognathic region, and generates AR data from the 3DCT data. Hereinafter, the 3D model of the stomatognathic region may be referred to as a base model, and the 3D model data of the stomatognathic region may be referred to as base data.

[0027] The AR model data created by the AR data generation module 213 may have a trigger for superimposing on a real image of a location (position information) type, a vision (image recognition) type, or a combination of these. The vision type may be any one of a marker type, a space recognition type, an object recognition type (three-dimensional recognition type), etc., or a combination of two or more of these. The AR model data is preferably an object tracking type AR model data that recognizes the shape of the object itself as a marker in real time without requiring a marker, and displays the AR model in an expanded form. For example, the AR model data may be configured to track a dental model having the same or similar form as the form of the AR model itself so that the orientation and dimensions are common.

[0028] The AR data generation module 213 processes the dental CT data by, for example, a three-dimensional computer-aided design (CAD) function and an image processing function. The image processing function typically includes a function unit that realizes binarization to multi-value processing, noise removal processing, edge enhancement processing, image recognition processing, 3D modeling, rendering processing, digital synthesis, and the like, and can be realized by, for example, 3DCG (three-dimensional computer graphics) software, etc. The image processing function may include an image analysis function, and can be realized by, for example, 3D modeling software (for example, Fiji: an open source platform for biological image analysis), OrtogOnBlender (open source 3D content creation software), etc. In addition, the AR data generation module 213 creates AR data from the 3DCT data by, for example, using an AR content creation function (for example, ARCore, ARKit, etc.).

[0029] The first generation module 213A of the AR data generation module 213 generates base data, which is 3D data related to tissues identified based on detection amount characteristics in the acquired dental CT data (an example of medical image data). The detection amount characteristics refer to information on the amount and distribution of signals emitted from a living body to create an image and received (detected) by a detector or the like. For example, X-ray photography, X-ray CT, and nuclear medicine diagnosis detect radiation emitted from a living body and visualize the detection signal as an image. Nuclear magnetic resonance imaging detects the nuclear magnetic resonance signal of water emitted from a living body, and ultrasonic tomographic imaging detects the reflected waves of ultrasonic waves transmitted to the living body and visualizes the detection signal. When using nuclear magnetic resonance imaging, image processing can be performed similarly based on the amount of hydrogen detected.

[0030] The first generating module 213A generates 3D data related to hard tissues identified by extracting information having a radiation absorption higher than a predetermined value from the dental CT data as base data. For example, the first generating module 213A creates 3D CT data of the maxilla, upper dentition, mandible, and lower dentition as base data from the dental CT data. The first generating module 213A also generates base data by performing processing according to the type of tissue on tissues such as the maxilla, upper dentition, mandible, and lower dentition.

[0031] The second generation module 213B of the AR data generation module 213 creates characteristic part data based on 3DCT data (an example of base data). The characteristic part data is data related to a 3D image corresponding to characteristic parts such as positions to be careful of and positions not to be treated when performing a treatment on a living body. The second generation unit, for example, identifies the characteristic part based on image processing of the 3DCT data, and generates 3D data related to the characteristic part as characteristic part data. The second generation unit, for example, identifies the characteristic part of the living body by image processing of the 3DCT data (base data) based on at least one of the above-mentioned detection amount characteristics and three-dimensional structure. In addition, the second generation module 213B, for example, performs processing on the identified characteristic part according to the type of the characteristic part to generate characteristic part data. A method for creating the characteristic part data will be described in detail later.

[0032] Specific examples of the characteristic parts in treatment of the oral cavity region include the mental foramen, mandibular canal (inferior alveolar canal), incisive canal, posterior alveolar canal (superior alveolar canal), greater palatine foramen, etc., which are related to the running positions of nerves and blood vessels. The characteristic parts may also include cortical bone, maxillary sinus walls, and other tissues. The characteristic parts may further include dentition (which may be predetermined teeth including natural teeth and dentures), implant bodies, etc. The positions and shapes of these characteristic parts differ from patient to patient, and thus the characteristic part data is patient-specific information, similar to dental CT data.

[0033] The AR data generation module 213 stores the 3D model data (base data) of the maxilla, upper dentition, mandible, lower dentition, upper gingiva, and lower gingiva generated as described above, and the characteristic part data, in the auxiliary storage device 202. At this time, the AR data generation module 213 stores these 3D model data, for example, in a file format that can be displayed in AR (for example, USDZ format or GBL format). In this way, the AR model data 500 is created. The AR data generation module 213 stores the base data and the characteristic part data in association with each other.

[0034] The dentition model generating module 214 generates dentition model data from dental CT data. The dentition model generating module 214 generates 3D printer data for forming a dentition model with a three-dimensional modeling device (3D printer 103) based on, for example, CT data of the patient's oral and maxillofacial region. The dentition model generating module 214 may generate dentition model data by using, for example, the above-mentioned base data, the AR model data 500, and data in an image file format generated by the AR data generating module 213. Although not limited thereto, for example, the dentition model generating module 214 may generate upper jaw model data from data of the upper jaw and upper dentition part, and generate lower jaw model data from data of the mandible and lower dentition part, for hard tissue. The dentition model generating module 214 may output, for example, a 3D surface of the upper jaw and upper dentition part as dentition model data. The 3D printer data is, for example, STL (Standard Triangulated Language) data. The dentition model generation module 214 is realized, for example, by a computer-aided design (CAD) function.

[0035] The teeth model generating module 214 stores the generated teeth model data as teeth model data 230 in the auxiliary storage device 202. The teeth model data 230 is data representing the shape of the teeth model to be modeled when the teeth model is modeled by the 3D printer 103. The teeth model generating module 214 also outputs the generated teeth model data to the 3D printer 103, for example. The teeth model generating module 214 outputs an instruction to the 3D printer 103 to create the teeth model data based on the teeth model data, for example. As a result, a teeth model corresponding to the teeth model data is created.

[0036] The NG area data generation module 215 creates the NG area data 800. The NG area data generation module 215 creates the NG area data 800 based on the above-mentioned AR model data created by the AR data generation module 213. The NG area data is data that indicates positions that require attention when performing treatment, positions where treatment is not performed, etc., and is data that is used as an evaluation criterion in the treatment evaluation related to the present technology. The NG area data is an example of check data in the present technology.

[0037] The NG area data may be, for example, position information representing a biological part such as the mental foramen, mandibular canal, posterior alveolar canal, and greater palatine foramen in the above-mentioned characteristic part data, and a region surrounding the biological part with a predetermined dimension. The dimension (separation distance) of the region surrounding the biological part may vary depending on the target biological part and the treatment content, for example, as shown in check data 700 (FIG. 7) described later. The NG area data may be, for example, position information representing the teeth in the above-mentioned upper dentition and lower dentition and a region surrounding the teeth with a predetermined dimension, or a region of a predetermined thickness on the side end (bone wall) of the buccal and lingual sides of the maxilla and mandible. The NG area data may be, for example, information representing a relatively inappropriate positional relationship of the guide with respect to the dentition or teeth. The NG area data generation module 215 generates the NG area data based on, for example, the characteristic part data and the dimensional information of the region surrounding the characteristic part included in the check data 700. The NG area data generation module 215 stores the created NG area data 800 in, for example, the auxiliary storage device 202 as the NG area data 800.

[0038] The NG area data generation module 215 may be configured to cooperate with the evaluation module 315 of the user terminal 102 to generate new NG area data (hereinafter, sometimes referred to as second NG area data) based on information indicating the user's treatment acquired by the user terminal 102. In this case, the generation process of the second NG area data by the NG area data generation module 215 may be similar to the generation process of the second NG area data by the evaluation module 315 of the user terminal 102, and a description thereof will be omitted here. The NG area data generation module 215 outputs the generated second NG area data to, for example, the auxiliary storage device 302 of the user terminal 102 as NG area data 800. In this embodiment, the NG area data 800 is generated by the NG area data generation module 215, but the NG area data 800 may be manually generated.

[0039] The NG area data generation module 215 may be configured to generate check information. The NG area data generation module 215 generates data that triggers the evaluation module 315 of the user terminal 102 to output information indicating that the treatment is inappropriate when the position of the tool based on the position information of the tool used in the treatment overlaps with the NG area based on the NG area data (an example of an area defined based on the characteristic part data).

[0040] The server-side evaluation module 216 evaluates the treatment by the user. The server-side evaluation module 216 cooperates with, for example, the evaluation module 315 of the user terminal 102 to evaluate the treatment by the user. The evaluation process in the server-side evaluation module 216 may be similar to the evaluation process in the evaluation module 315 described below, and a description thereof will be omitted here. The server-side evaluation module 216 cooperates with the evaluation module 315 of the user terminal 102 to output (display) the evaluation result on the display (an example of the output device 305; the same applies below) of the user terminal 102.

[0041] The treatment support data output module 217 outputs treatment support data. The treatment support data is data used to support treatment (including training). The treatment support data output module 217 outputs, for example, base data and characteristic part data as treatment support data. Furthermore, when outputting the base data and the characteristic part data, the treatment support data output module 217 outputs them in association with each other. Furthermore, the treatment support data output module 217 may output, for example, either the base data or the characteristic part data as treatment support data. The treatment support data output module 217 outputs the treatment support data to, for example, the auxiliary storage device 202. Note that the treatment support data output module 217 may be a module that outputs the base data, the characteristic part data, and the like, and is not limited to a configuration that outputs these data as treatment support data.

[0042] [User device] 3 is an example of a hardware configuration of the user terminal 102. The user terminal 102 is a terminal operated by a user who uses the dental treatment support system 100, and is an example of a dental treatment support device in the present technology. The user terminal 102 is configured by a terminal such as a smartphone, a tablet, a computer with AR glasses, a computer with a head-mounted display, or a head-mounted AR terminal.

[0043] The user terminal 102 includes a main memory device 301 and an auxiliary memory device 302. The user terminal 102 also includes a processor 303, an input device 304, an output device 305, a camera 306, and a communication control unit 307 as described above. The camera 306 may be an imaging device built into a smartphone or a tablet, or may be an imaging device built into a wearable device such as AR glasses or a head-mounted display. The output device 305 may be a display device (display) provided in the smartphone or tablet, or may be a display device (display) in the AR glasses or a head-mounted display.

[0044] The main memory device 301 stores programs and applications such as a management module 311, a reception module 312, a data acquisition module 313, a display module 314, and an evaluation module 315. The processor 303 executes these programs and applications to realize each functional element of the user terminal 102.

[0045] The auxiliary storage device 302 stores information necessary for the operation of the user terminal 102. For example, the auxiliary storage device 302 stores user information 400, AR model data 500, training data 600, check data 700, NG area data 800, etc. The auxiliary storage device 302 may be a part or all of this information stored in the auxiliary storage device 202 of the management server 101.

[0046] A brief explanation will be given below of each functional element of the user terminal 102. The specific operation of each functional element will be described later. The management module 311 controls the basic operation of the user terminal 102 when using the dental treatment support system 100. The management module 311 cooperates with the user terminal management module 211 of the management server 101, for example, to output (display) a login page for the dental treatment support site, a menu page showing training contents for treatment on a dentition model, and the like, on the display of the user terminal 102.

[0047] The reception module 312 receives an operation of the user terminal 102 by the user. The reception module 312 receives, for example, a selection of treatment contents for a dentition model. The reception module 312 receives, for example, a selection operation for a treatment content that the user wishes to train in from among the treatment training contents displayed on a menu page.

[0048] The data acquisition module 313 acquires information corresponding to the selected treatment content. The data acquisition module 313 cooperates with the management module 311 of the management server 101 to acquire, for example, AR model data, NG area data, and check data corresponding to the selected treatment content. The data acquisition module 313 stores the acquired AR model data, NG area data, and check data, for example, in the auxiliary storage device 302, as AR model data 500, check data 700, and NG area data 800, respectively. Furthermore, the data acquisition module 313 inputs training data 600, which will be described later, based on the acquired data.

[0049] The display module 314 displays the AR model generated based on the AR model data on a display. The display module 314 displays the AR model generated based on the AR model data in association with the dental arch model being imaged. The display module 314 may be configured to display the AR model in a form that can be switched between AR content and an object in response to a user's selection, for example.

[0050] The evaluation module 315 evaluates the treatment performed on the dentition model and outputs the evaluation result. The evaluation module 315 evaluates the treatment performed on the dentition model by the user based on, for example, the check data 700, and outputs the evaluation result. In addition, in the treatment evaluation, the evaluation module 315 performs image recognition and generates second NG area data based on the treatment imaging data captured by the camera 306. The evaluation module 315 stores the generated second NG area data in, for example, the auxiliary storage device 202 as NG area data 800.

[0051] [Information DB] FIG. 4 is an example of user information 400. The user information 400 is information about a user who uses this dental treatment support system 100. The user information 400 includes, for example, a user ID, a user display ID, a user name, an address, payment information, password information, and personal identification information, and values ​​such as those shown as examples in sample values ​​420 are input for each field name 410. The outline of the field name 410 and the sample value 420 are explained in an outline section 430. The contents of the user information 400 are not limited to this example.

[0052] FIG. 5 is an example of the AR model data 500. The AR model data 500 is information about an AR model of a patient who is a target of treatment. The AR model data 500 includes information such as an AR model ID, a user ID, a patient ID, a mandible model, a lower dentition model, a lower gum model, and a mandibular canal model, and values ​​such as those shown as sample values ​​520 are input to the respective field names 510.

[0053] The AR model ID is a code given to identify the AR model of the patient who is to be operated by the user. The patient ID is a code given to identify the patient who is the origin of the AR model. The mandible model, the lower dentition model, the lower gum model, and the mandibular canal model are AR models of the mandible, the lower dentition, the lower gum, the mandibular canal, etc., which respectively constitute the AR model, and information on these AR models is stored, for example, in a file with a file name shown in the value 520 column. Note that the value 520 column may store the storage destination of these AR model files. However, the contents of the AR model data 500 are not limited to this example.

[0054] FIG. 6 is an example of training data 600. The training data 600 is information about data used for training provided for each treatment content. The training data 600 includes information such as a training ID, an AR model ID, a checkpoint ID, and an NG area ID, and values ​​such as those shown as sample values ​​620 are input for each field name 610.

[0055] The training ID is a code given to identify the training provided for each treatment content. The treatment content may include, for example, steps such as tooth extraction, incision, drilling, and implant insertion. The checkpoint ID is a code given to identify information related to the treatment content (checkpoint) to be checked, which is set for each training or for each treatment step of the training. The NG area ID is a code given to identify information indicating the NG area for each AR model. The treatment content may differ depending on the patient. In addition, the spread of the NG area may differ depending on the patient. The ID of the training performed for each treatment content for the AR model and the ID of the checkpoint and the NG area used in the training are linked by this training data 600.

[0056] FIG. 7 is an example of check data 700. The check data 700 is information on the contents to be checked for each checkpoint ID. For example, in the check data 700, values ​​such as those exemplified by sample values ​​720 are input for field names 710 including detailed checkpoint IDs (detailed CPIDs). The contents of the information indicated by the sample values ​​720 are written in the summary 730 column. For example, the check data 700 includes information on treatment conditions that must be met when performing treatment. For example, the check data 700 includes information indicating areas that require attention and areas that should not be treated, based on the characteristics of a living body. The contents of the check data 700 are not limited to this example. Although not specifically described, the check data 700 can be set for each treatment content or each treatment step, for example.

[0057] The check data 700 may include, for example, information regarding the tool to be used. The information regarding the tool may include, for example, a tool ID for identifying the tool, information regarding the 3D shape of the tool, and information such as the type of tool to be used for each treatment step. The check data 700 may also include, for example, information regarding the treatment position and treatment depth of the tool, the angle (posture) of the tool, and the like. As described above, the NG area data 800 shown below is an example of the check data 700.

[0058] FIG. 8 is an example of NG area data 800. The NG area data 800 is information about the NG area. The NG area data 800 is, for example, information about the position of one or more NG areas provided for each AR model identified by an AR model ID. The NG area may be provided for each detailed checkpoint ID so as to correspond to the detailed checkpoint ID indicated in the check data 700. For example, the NG area may be provided so as to include a case where the treatment condition (check content) to be satisfied at the checkpoint identified by each detailed checkpoint ID is not satisfied. In this embodiment, the information about the position of the NG area is expressed, for example, as a three-dimensional coordinate group (hereinafter, may be simply referred to as a "coordinate group"). However, the position information in the present technology is not limited to this. In the NG area data 800, for example, values ​​such as those exemplified by the sample value 820 are input for the field name 810. In addition, the content indicated by the sample value 820 is described in the summary 830 column.

[0059] [Dental treatment support method] In this embodiment, the dental treatment support system 100 is used to evaluate treatment (training) on ​​a dental arch model. [Training Preparation] First, preparation for training will be briefly described. Fig. 9 shows an example of a treatment support data generation flow 900. Fig. 16 shows an example of a CT image 1600 based on CT data.

[0060] In step S910, the dental CT acquisition module 212 of the management server 101 acquires dental CT data of the patient captured in advance, for example, from the user terminal 102. As shown in Fig. 16, the reconstructed CT image generally includes a portion that is not required for the dentition model, in other words, a portion that is not required for the AR data. Therefore, in step S920, the first generation module 213A of the AR data generation module 213 generates base data on the hard tissue of the oral cavity region of the patient based on the acquired dental CT data.

[0061] FIG. 10 is an example of a base data generation flow 1000 corresponding to step S920. In step S1010, the first generation module 213A identifies hard tissues by extracting information with radiation absorption higher than a predetermined value based on the CT data. Specifically, the first generation module 213A creates a 3D CT image (e.g., voxel data) of the oral and maxillofacial region from the acquired dental CT data, cuts out a portion including the maxilla and upper dentition and a portion including the mandible and lower dentition, and then extracts the maxilla, upper dentition, mandible, lower dentition, etc., which are hard tissues, based on the difference in feature amount (here, CT value, X-ray transmittance, and X-ray absorption amount) for each biological tissue (segmentation). Although it cannot be generalized because it varies depending on the imaging device, resolution, reconstruction method, etc. and individual differences, the first generating module 213A extracts information on parts with radiation absorption of approximately 100 HU or more (e.g., 250 HU or more) and less than 2500 HU (e.g., 2000 HU or less, 1000 HU or less) using a CT value expressed as a relative value with the radiation absorption rate of air being -1000 HU and the radiation absorption rate of water being 0 HU, thereby extracting hard tissue. At this time, the first generating module 213A may delete, for example, a jaw part, a cervical vertebrae part, and other artifact parts that are not necessary for the dentition model from the 3D CT image.

[0062] The first generating module 213A can specify the tooth part by extracting information having a higher radiation absorption than a threshold value, for example, a predetermined radiation absorption lower than the radiation absorption of teeth and higher than the radiation absorption of bone parts (maxilla and mandible).The first generating module 213A can specify the jaw part by extracting information having a higher radiation absorption than a threshold value, for example, a predetermined radiation absorption lower than the radiation absorption of jaw parts (maxilla and mandible) and higher than the radiation absorption of soft tissues (other than hard tissues such as gums, membranes, nerves, blood vessels, etc.).

[0063] At this time, the first generation module 213A identifies the maxilla and mandible as separate elements. The first generation module 213A may extract the mandible and lower dentition by, for example, selecting continuous (adjacent) voxels from the bottom end of the hard tissue of the 3DCT image among the cubes (also called pixels or voxels) that constitute the 3DCT image. In this case, the remaining parts excluding the extracted mandible and lower dentition can be the maxilla and upper dentition. The first generation module 213A may delete the upper part of the maxilla as a part unnecessary for the dentition model, for example, based on conditions defined in a pre-prepared maxilla dentition model template.

[0064] The first generation module 213A can identify the presence or absence of missing teeth and each tooth based on the CT data of the tooth part. The first generation module 213A can execute the identification of the presence or absence of missing teeth and each tooth by, for example, a pattern recognition method, a difference image calculation method, a template matching method, or the like. For example, the first generation module 213A can identify the presence or absence of missing teeth and each tooth by comparing the tooth part in the CT data with a tooth row template prepared in advance. As the tooth row template, for example, a tooth row template of a human tooth row (when there is no missing tooth) consisting of a total of 32 teeth including a third molar, with three types of dental arch shapes of approximately U-shape, V-shape, and square can be used. The first generation module 213A can grasp the presence or absence of missing teeth and identify (segment) each tooth by comparing the CT data of the tooth part with the tooth row template.

[0065] Although not limited thereto, for example, the first generating module 213A extracts information on a portion where the CT value corresponding to the radiation absorbance is approximately -50 HU or more (e.g., 0 HU or more) and less than 60 HU (e.g., 70 HU or less, 60 HU or less) to extract soft tissue. Also, the first generating module 213A may be configured to extract metal tissue (implant body, metal prosthesis, etc.) by extracting information on a portion where the CT value corresponding to the radiation absorbance is approximately 3000 HU or more (e.g., approximately 3500 to 4000 HU).

[0066] The first generation module 213A may be configured to identify the cancellous bone portion by extracting the porous portion of the hard tissue. For example, the first generation module 213A identifies the cancellous bone portion as the hard tissue present between a plurality of nutrient holes found in the cancellous bone of the maxilla and mandible. The first generation module 213A may identify, for example, the hard tissue of a predetermined thickness or less present inside the maxilla and mandible as the cancellous bone portion.

[0067] Furthermore, the first generation module 213A may be configured to identify (segment) from the CT data, for example, adult compact bone, child compact bone, adult enamel, child enamel, adult adipose tissue, child adipose tissue, adult muscle tissue, child muscle tissue, adult skin tissue, child skin tissue, adult cancellous bone tissue, and child cancellous bone tissue based on differences in radiation absorption for each biological tissue.

[0068] Furthermore, when the acquired dental CT data is so-called dental cone beam CT data, since information on soft tissue is not acquired, the first generation module 213A can omit the extraction of the hard tissue. In this case, the first generation module 213A can create 3D CT data by deleting the jaw part, cervical vertebrae part, and other artifact parts that are not necessary for the dentition model from the dental cone beam CT data.

[0069] Then, in step S1020, the first generating module 213A generates 3DCT data for the hard tissue based on the data related to the hard tissue identified as described above. For example, the first generating module 213A can store the data related to the identified hard tissue in a DICOM format, a voxel file format (for example, FAV format), an intermediate data format (for example, STL format, STEP format, OBJ format), other 3D file formats, etc. At this time, the first generating module 213A can assign identifiers such as tags to the identified teeth, maxilla, mandible, etc. Also, the first generating module 213A may store the data related to the identified hard tissue as an image file (for example, Tiff format, PNG format, etc.). This generates base data. Furthermore, the first generating module 213A creates AR model data 500 based on the 3DCT data, for example, by using an AR content creation function. For example, the first generation module 213A stores the 3D CT data related to the hard tissue identified as described above in a format that allows AR display, such as the USDZ format or the GBL format, etc. In this way, the AR model data 500 is created.

[0070] To explain further, the first generation module 213A performs processing according to the type of tissue on tissues such as the maxilla, upper dentition, mandible, and lower dentition extracted from the CT data, to generate base data. For example, in the alveolar crest of the extracted maxilla, cavities in which teeth (in this example, the upper dentition, typically the tooth root) are placed (inserted) exist corresponding to each tooth. For example, if this cavity is small or large (for example, if the cavity is not large enough to insert the tooth or is too large), the first generation module 213A performs processing to create a cavity of a shape corresponding to the tooth so that the tooth can be inserted and held. The processing to create a cavity is, for example, processing to delete an image of an area in which a cavity is to be created. Similarly, in the alveolar crest of the extracted mandible, cavities in which teeth (in this example, the lower dentition, typically the tooth root) are inserted exist corresponding to each tooth. If the cavity is small or large (e.g., if the cavity is too large or too small to insert the tooth), the first generating module 213A performs a process to create a cavity of a shape corresponding to the tooth so that the tooth can be inserted and held. This process may be, for example, a process to remove (Subtract) a part of the jawbone that overlaps with the tooth. This process may also be a process to add (Add) a gap between the jawbone and the tooth if the gap is larger than a predetermined volume or dimension.

[0071] Also, for example, in the extracted upper and lower teeth rows, when each tooth is not properly identified and, for example, when the teeth are connected, the first generation module 213A performs a process of separating the teeth so that they are not connected. More specifically, for example, the first generation module 213A performs a process of deleting an image of a portion where the teeth are connected. Or, for example, one tooth and an adjacent tooth are properly segmented as different teeth.

[0072] Also, for example, the first generating module 213A performs a process of creating a cavity when the area of ​​the root canal part is closed in the extracted upper or lower dentition, in other words, when there is no cavity in the area of ​​the root canal part. Specifically, for example, in each tooth, the part where the CT value corresponding to the radiation absorbance is higher than a predetermined threshold is extracted as the root canal part. Then, in each tooth, the part where the CT value corresponding to the radiation absorbance is lower than the predetermined threshold is extracted as the cavity, in other words, the dental pulp part.

[0073] Also, for example, the first generating module 213A may delete the jaw portion, cervical vertebra portion, and other artifact portions that are unnecessary for the dentition model, as described above. Examples of artifact portions include those caused by reflected light from metal attached to teeth. When the first generating module 213A detects an artifact portion caused by reflected light in the CT data, it performs a process of deleting the image of the detected artifact portion.

[0074] Fig. 17 is an example of an AR model. In Fig. 17, from the viewpoint of ease of viewing the drawing, only the left side of the mandible of the AR model 1700 is shown, which is obtained by dividing the mandible by the median plane. The AR model 1700 includes, for example, a mandible model 1710 and a tooth model 1720. As shown in this AR model 1700, the related patient is missing a first molar.

[0075] Returning to FIG. 9, in step S930, the second generation module 213B of the AR data generation module 213 generates feature part data based on the 3D CT data (base data).

[0076] FIG. 11 shows an example of a characteristic portion generation flow 1100 corresponding to step S930. In step S1110, the second generation module 213B acquires 3DCT data (base data) of the identified teeth, maxilla, mandible, etc. In step S1120, the second generation module 213B identifies feature parts by image processing of the base data based on the detection amount characteristics or three-dimensional structure. Details of the processing in step S1120 will be described later with reference to FIG. 12. In step S1130, the second generation module 213B generates feature part data based on the extracted feature parts.

[0077] [Identifying characteristic parts based on detected quantity characteristics] FIG. 12 is an example of a hollow portion identification flow 1200 corresponding to step S1120. First, in step S1210, the second generation module 213B identifies an area with low radiation absorption by image processing based on radiation absorption characteristics (an example of detection amount characteristics) of data (base data) indicating hard tissue. Then, in step S1220, the second generation module 213B identifies this area with low radiation absorption as a hollow part as a characteristic part. Examples of hollow parts identified in this way include the mandibular canal, incisive canal, posterior alveolar canal, greater palatine foramen, maxillary sinus wall, mental foramen, mandibular foramen, and greater palatine foramen.

[0078] Furthermore, the second generating module 213B identifies a longitudinal hollow portion among the above-mentioned hollow portions as a tubular portion. The "longitudinal shape" of a hollow portion that can be identified as a tubular portion is not limited to this, but may be, for example, a diameter (e.g., maximum diameter) of 0.5 mm or more (typically, 10 mm or less) and an aspect ratio of 3 or more (e.g., 5 or more). Examples of such tubular portions include the mandibular canal, the incisor canal, and the posterior alveolar canal.

[0079] FIG. 18 is another example of the AR model 1800. The second generation module 213B shows a state in which, for example, a surface rendering process is applied to the AR model 1700 of FIG. 17. In contrast, the second generation module 213B shows a state in which, for example, a volume rendering process is applied to the AR model 1800 of FIG. 18. The surface rendering process displays the surface of a mass above a certain threshold in a three-dimensional manner, even though it is a two-dimensional image, and the volume rendering process changes the opacity of the mass so that the contents can be seen. In the AR model 1800 of FIG. 18, the mandibular canal 1850 and the mandibular foramen 1840 can be more clearly confirmed. In this way, the second generation module 213B applies a volume rendering process to the base data, and thus the hollow portion and the tubular portion can be accurately identified.

[0080] FIG. 19 is another example of the AR model 1900. The second generation module 213B shows a state in which, for example, a surface rendering process has been applied to the AR model 1700 of FIG. 17. In contrast, the second generation module 213B shows a state in which, for example, a sharpening process has been applied to the AR model 1900 of FIG. 19. The sharpening process displays the contour so that it appears clearer by increasing the brightness difference and the hue difference. In the AR model 1900 of FIG. 19, the mandibular canal 1950 can also be confirmed more clearly. In this way, the second generation module 213B applies the sharpening process to the base data, and thus the hollow portion and the canal portion can be identified with high accuracy.

[0081] [Identification of characteristic parts based on detected quantity characteristics and three-dimensional structure (position information)] (mandibular canal) In the AR model 1800, the first generation module 213A extracts the mandible and teeth of a living body as hard tissues, and the second generation module 213B identifies a tubular portion of the mandible extending in the anterior-posterior direction (mesial-distal direction) below the teeth 1820 as the mandibular canal 1850.

[0082] The position information for identifying the mandibular canal 1850 may be one or more of the following: A tube portion that extends from the mental foramen 1830 (described later) to the mandibular foramen 1840 (not clearly shown because it opens on the tongue side) while drawing a loop (anterior loop) from the anterior lower medial to the posterior lower medial. A tube extending from the mandibular foramen 1840 (described later) toward the lower front (mesial direction) inside the mandible A tube portion that extends backward inside the mandible from the mental foramen 1830 (described later) while drawing a loop (anterior loop) from the anterior lower medial to the posterior lower medial. A tube that extends backwards inside the mandible from the mental foramen (described later) while drawing a loop (anterior loop) from the anterior inferior medial to the posterior inferior medial A canal extending backwards (distal) at the level of the middle of the mandibular body, below the first and second premolars

[0083] (inferior alveolar nerve, artery and vein) In addition, the inferior alveolar nerve, artery, and vein run inside the mandibular canal. Therefore, the first generation module 213A extracts the mandible and teeth of a living body as hard tissues, and the second generation module 213B specifies a canal part of the mandible extending in the anterior-posterior direction (mesio-distal direction) below the teeth 1820 as the running position of the nerve and / or blood vessel. In addition, the second generation module 213B processes the identified nerve to make it dense. In addition, since the identified nerve is hollow, the second generation module 213B processes the nerve to fill the hollow, for example, to make it gap-free. More specifically, the second generation module 213B performs image processing to fill the hollow of the identified nerve. That is, the second generation module 213B estimates and extracts the hollow part as a nerve or an artery or vein, and generates data of the nerve or artery or vein having a shape corresponding to the shape inside the cavity.

[0084] (Posterior alveolar canal) The second generation module 213B identifies the posterior alveolar canal (superior alveolar canal) by determining one or more of the following: A canal or cavity located within the inner wall of the maxillary sinus within the maxilla The distance from the floor of the maxillary sinus to the maxillary sinus is (a) 8.5±4.8 mm upward at a position 5 mm forward from the zygomatic ridge, (b) 9.6±4.7 mm upward at the position of the zygomatic ridge, (c) 9.5±3.6 mm upward at a position 5 mm backward from the zygomatic ridge, and (d) 10.1±3.9 mm upward at a position 10 mm backward from the zygomatic ridge.

[0085] The "maxillary sinus" refers to the outer lower cavity that corresponds to the inside of the cheek, among the four cavities called paranasal paranasal sinuses that connect to the left and right nostrils. The "zygomatic ridge" refers to a cross section parallel to the frontal plane (any plane that divides a living body into ventral and dorsal sides (anterior and posterior in humans)) that passes through the outer end of the maxillary sinus on a plane (ZMS plane) that is parallel to the occlusal plane that passes through the lowest point of the zygomaticomaxillary suture (the joint between the maxilla and the zygomatic bone).

[0086] (Posterior superior alveolar artery) The first generation module 213A extracts the maxilla and teeth of a living body as hard tissues, and the second generation module 213B estimates this posterior alveolar canal as the running position of a blood vessel (posterior superior alveolar artery) and generates the posterior superior alveolar artery in this part. In other words, the artery data is generated by estimating that there is an artery corresponding to the shape of the canal inside the canal.

[0087] (Apical periodontitis) The second generation module 213B detects the apical lesion based on the dental CT data. Specifically, for example, the second generation module 213B identifies and extracts the hollow part inside each tooth detected as hard tissue as the root canal and / or dental pulp. The second generation module 213B also generates lesion data having a shape corresponding to the extracted hollow part. The second generation module 213B also identifies, as the tooth root, one or more protruding parts that protrude upward from the crown of each tooth on the maxillary bone side, and one or more protruding parts that protrude downward from the crown of each tooth on the mandibular bone side, among each tooth detected as hard tissue. This makes it possible to grasp the positions of the root canal, dental pulp, and dental root of the tooth with periodontitis of the patient, for example, when a root canal treatment is performed to remove the infected and necrotic nerve (dental pulp) and bacteria from the inside of the tooth (inside the root canal) when a patient develops apical periodontitis.

[0088] When a dark area (area with relatively low radiation absorption) of a certain size is detected inside the soft tissue (alveolar bone, periodontal ligament) around the tip (apex) of each tooth root thus identified, the second generation module 213B can estimate and extract the area with low radiation absorption as an apical lesion. When a dark area (area with relatively low radiation absorption) of a certain size is detected inside the soft tissue (alveolar bone, periodontal ligament) at the position of the hole (apical foramen) at the tip of the root canal, the second generation module 213B may estimate and extract the area with low radiation absorption as an apical lesion. Also, for example, the second generation module 213B can identify the area identified as the apical lesion as an area to be cut and removed by treatment.

[0089] (Root cyst) The second generation module 213B detects a radicular cyst in the same manner as in the case of apical periodontitis. That is, the second generation module 213B, for example, identifies and extracts a hollow part inside each tooth detected as hard tissue as a root canal and / or dental pulp. The second generation module 213B also generates cyst data having a shape corresponding to the extracted hollow part. The second generation module 213B also identifies, as the tooth root, one or more protruding parts protruding upward from the crown of each maxillary tooth, and one or more protruding parts protruding downward from the crown of each mandibular tooth, among each tooth detected as hard tissue.

[0090] When a dark area (area with relatively low radiation absorption) of a certain size is detected inside the soft tissue (alveolar bone, periodontal ligament) around the tip (apex) of each tooth root thus identified, the second generation module 213B estimates and extracts the area with low radiation absorption as a radicular cyst. When a dark area (area with relatively low radiation absorption) of a certain size is detected inside the soft tissue (alveolar bone, periodontal ligament) at the position of the hole (apical foramen) at the tip of the root canal, the second generation module 213B may estimate and extract the area with low radiation absorption as a radicular cyst. This makes it possible to grasp the area where the radicular cyst was formed, for example, when a patient develops a radicular cyst and a cyst is formed at the tip (apex) of the tooth root, causing abnormal proliferation of cells on the surface of the tooth root in a bag-like shape. In addition, for example, the second generation module 213B can identify the area identified as a radicular cyst as an area to be removed by surgery.

[0091] The second generating module 213B adjusts the display range of the CT value in the CT image based on the 3DCT data by narrowing the WW (Window width) (for example, 256) and adjusting the WL (Window level) to match the soft tissue (for example, about 0 to 60). Generally, a root cyst has a lower radiation absorption than an apical periodontitis, so the second generating module 213B acquires the CT value of the region with the relatively low radiation absorption for the image adjusted in this way, and estimates that the CT value is a cyst when it is equal to or greater than a predetermined value. Alternatively, the second generating module 213B may estimate that the cyst is a cyst when the difference in radiation absorption at the boundary between the region with the relatively low radiation absorption and the surrounding bone is equal to or greater than a predetermined value. This makes it possible to distinguish between apical periodontitis and radiation absorption.

[0092] Moreover, the second generating module 213B processes the identified radicular cyst to make it dense. In addition, since the identified radicular cyst is a cavity, the second generating module 213B processes the cavity, for example, by filling it so that there are no gaps. More specifically, the second generating module 213B performs a process to fill the cavity of the identified radicular cyst, for example. Such a process can be regarded as a process of artificially creating a focus (or lesion).

[0093] (tumor) The second generation module 213B can identify and extract a tumor formed in the gingival portion. The second generation module 213B segments data corresponding to the shape of the extracted tumor to generate separated tumor data. The tumor may be present around the tooth or may be independent without the presence of the tooth. When the tumor is present around the tooth, the second generation module 213B can extract the tumor by estimating the tumor as a tumor when an opaque image with a clear boundary with the surrounding tissue is observed in a CT image of the gingival portion around the tooth, as in the case of apical periodontitis or radicular cyst. When the tumor is not present around the tooth, the second generation module 213B can extract the tumor by estimating the tumor as a tumor when an opaque image with an unclear boundary with the surrounding tissue is observed. In addition, for example, the second generation module 213B can identify the area identified as a tumor as a region to be removed by surgery.

[0094] The second generation module 213B classifies the above-mentioned region (lesion) with relatively low radiation absorbance as either ameloblastoma or odontogenic keratocyst. As a method for distinguishing between ameloblastoma and odontogenic keratocyst, the second generation module 213B distinguishes between ameloblastoma and odontogenic keratocyst, for example, by comparing the morphology of the lesion with the tissue surrounding the tooth. More specifically, the second generation module 213B distinguishes between ameloblastoma and odontogenic keratocyst by determining one or more of the following regarding radiation absorbance in a CT image in a soft tissue drawing mode (for example, CT value = 0 to 60):

[0095] When an area of ​​relatively high radiation absorption due to an accumulation of keratinized matter (high-density components) is found within an area of ​​relatively low radiation absorption (lesion), the second generation module 213B can estimate and extract the area of ​​low radiation absorption as an odontogenic keratinocyte. When two or more of the following (a) to (c) are satisfied, the second generating module 213B can estimate and extract the region with low radiation absorption as ameloblastoma. (a) The aspect ratio (ratio of the long axis to the short axis perpendicular to the long axis) of the maximum cross-section of an area (lesion) with relatively low radiation absorption exceeds 0.5. (b) The buccal side of the area (lesion) with relatively low radioabsorption exhibits a bulge accompanied by thinning of the cortical bone. (c) The edges of areas (lesions) with relatively low radiation absorption exhibit a scallop-like shape. Odontogenic tumors tend to occur in young people, and most of them are benign, but they can recur or turn malignant, so early diagnosis and treatment are desirable. Odontogenic cysts and odontogenic tumors that develop from tooth tissues present relatively similar clinical and image findings to the human eye. However, with the above configuration, it is possible to distinguish between odontogenic cysts and odontogenic tumors based on image processing of the clinical and image findings. Furthermore, the second generation module 213B processes the identified tumor to make it dense. In addition, since the identified tumor is a cavity, the second generation module 213B processes the tumor to fill in the cavity, for example, to make it gap-free. More specifically, the second generation module 213B performs a process to fill in the cavity of the identified tumor, for example. Such a process can be regarded as a process to artificially create a focus (or a lesion).

[0096] [Identification of characteristic parts based on 3D structure] FIG. 13 is an example of a surface morphology feature portion identification flow 1300 corresponding to step S1120. First, in step S1310, the second generation module 213B specifies the surface morphology by image processing based on the three-dimensional structure of data (base data) showing the hard tissue. Then, in step S1320, the second generation module 213B specifies a portion different from the main portion of the hard tissue as a characteristic portion based on the surface morphology. Here, the "main portion" can be, for example, a portion having a morphology that occupies more than 75 area% (typically 80 area% or more) of the surface of the hard tissue, and the "characteristic portion" can be, for example, a portion having a morphology that occupies less than 25 area% (typically 20 area% or less, for example 15 area% or less) of the surface of the hard tissue. The "main portion" can also be, for example, 85 area% or more, 90 area% or more, or 95 area% or more. The "characteristic portion" can also be, for example, 10 area% or less, 5 area% or less.

[0097] More specifically, the second generation module 213B detects a closed contour by edge detection, and detects the inside of the closed contour as a concave (hole) or convex if the area of ​​the inside is less than 25% and the internal tissue of the closed contour is sparse or dense. When identifying the surface morphology of data (base data) indicating hard tissue, for example, as shown in a comparison between FIG. 17 and FIG. 18, the second generation module 213B performs a surface rendering process on the base data to make it easier to detect concave (hole) or convex parts. The second generation module 213B may detect holes provided on a flat surface by utilizing the structural feature that the brightness, etc. changes significantly around the hole (edge) in this way.

[0098] In addition, the second generation module 213B extracts a hole as a characteristic part when, for example, a part of the surface morphology where the degree of unevenness is high (for example, a part with a large inward recession depth) is compared with a part where the degree of unevenness is low (for example, a part with low surface roughness) and the area of ​​the part is less than 20% by area.

[0099] Furthermore, the second generating module 213B extracts a hole (through hole) as a characteristic part when, for example, a concave part of the surface morphology penetrates from one surface to the opposite surface and the area of ​​the region is less than 20% by area. Examples of holes identified in this way include the mental foramen, mandibular foramen, and greater palatine foramen.

[0100] (mental foramen) The second generation module 213B identifies the mental foramen 1830 by determining whether any one or more of the following conditions are met. Furthermore, if the outlet of the identified mental foramen is blocked, the second generation module 213B processes the area of ​​the outlet so that a hole is formed. For example, if the area of ​​the outlet of the mental foramen is smaller than a predetermined size, the second generation module 213B processes the area of ​​the outlet of the mental foramen so that it is equal to or larger than the predetermined size. The hole in the mandible, located below the teeth, can be identified as the mental foramen. A hole in the mandible located 2-4 cm (e.g., 2-3 cm) distal to the midline can be identified as the mental foramen. The foramen in the mandible, located below the first and / or second premolars and approximately midway between the alveolar crest and the lower border of the mandible, can be identified as the mental foramen.

[0101] (mandibular foramen) The position information for the second generation module 213B to identify the mandibular foramen can be one or more of the following: When the outlet of the identified mandibular foramen is blocked, the second generation module 213B performs processing so that a hole is opened at the outlet of the mandibular foramen, similar to the mental foramen. A foramen in the mandible that opens onto the lingual surface of the ramus and is located at the height of the occlusal plane in the vertical direction of the ramus may be identified as the mandibular foramen. The mandibular ligule, a plate-like structure that points upward and inward, is located at the anterior edge of the mandibular foramen, and the upper end of the mandibular ligule is located approximately 1 cm above the occlusal plane. The occlusal plane is a plane that includes three points: the contact points of the incisal edges (tips) of the mandibular central incisors and the distal buccal cusp tips of the left and right mandibular second molars.

[0102] (greater palatine foramen) The second generation module 213B identifies the greater palatine foramen by determining whether any one or more of the following conditions are met. If the outlet of the identified greater palatine foramen is blocked, the second generation module 213B performs processing so that a hole is opened at the outlet of the greater palatine foramen, similar to the mental foramen. - A foramen in the maxilla, located on the line connecting the left and right third molars - A hole located at the transition between the alveolar apex and the hard palate, just anterior to the posterior edge of the hard palate -It is a hole located about 15 mm posterior (distal) from the midline (median annular suture) and opens anteriorly, inferiorly, and medially. -CT data reveals clear bone permeability holes in the above areas. · Placed inside the root of the third or second molar in the roof of the palate.

[0103] (Greater palatine nerve) The second generation module 213B identifies the greater palatine canal as a canal connecting the oral cavity and the pterygopalatine fossa. The first generation module 213A then extracts the maxilla and teeth of the living body as hard tissues, and the second generation module 213B identifies this greater palatine canal as the running position of the greater palatine nerve (i.e., the greater palatine nerve). The second generation module 213B processes the identified greater palatine nerve to make it dense. In addition, since the identified greater palatine nerve is a cavity, the second generation module 213B processes the cavity to fill it in, for example, to make it gap-free. More specifically, the second generation module 213B adds an image such as a polygon to the cavity of the identified greater palatine nerve to fill the cavity.

[0104] The second generation module 213B of the AR data generation module 213 records the generated feature part data in the auxiliary storage device 202. The second generation module 213B can store the generated feature part data as, for example, 3DCT and AR data.

[0105] 9, in step S940, the NG area data generation module 215 creates NG area data from the 3D model data for the dentition model. As described above, the NG area data generation module 215 processes the characteristic part data related to the mandibular canal and generates the area surrounding the mandibular canal and its periphery with a thickness of 3 mm as NG area data. Although not limited to this, the NG area data generation module 215 generates NG area data by, for example, moving the outer surface of the characteristic part by a predetermined dimension (e.g., 3 mm) toward the outside in the tangential direction of the outer surface.

[0106] 22 is an example of an image in which the AR model 2100 of the mandible and the NG area 2260 are superimposed. The AR model 2100 includes a mandible model 2210 and a tooth model 2220. The NG area 2260 is one of the NG areas displayed based on the NG area data created by the NG area data generation module 215. The NG area 2260 has an area that combines a portion corresponding to the mandibular canal and a portion 3 mm around it. The tool 2240 is, for example, a treatment sample that shows an ideal treatment example for the mandible model 2210.

[0107] The NG area data generation module 215 can generate NG area data related to not only the NG area 2260 shown in Fig. 22 but also other multiple checkpoints. For example, the NG area data generation module 215 processes the tooth model data and generates NG area data for the tooth and the area surrounding it with a thickness of 1.5 mm. The tooth may be any of a natural tooth, a denture, and an implant.

[0108] The NG area data generating module 215 may also generate NG area data indicating an NG area 2130 having an area that combines a portion corresponding to a tooth and a portion 1.5 mm around the tooth. The NG area data generating module 215 may generate NG area data not only for one (one type) NG area 2130 but also for a plurality of other NG areas.

[0109] (Check information) Although not specifically illustrated, the NG area data generating module 215 may be configured to function as a check information generating unit in the present technology. That is, the NG area data generating module 215 generates check information for outputting information indicating that the treatment is inappropriate when the position of the tool based on the position information of the tool used in the treatment overlaps with an area (typically, the NG area and the second NG area) defined based on the characteristic portion data.

[0110] Such check information is not limited to this, but may be, for example, a mechanism (e.g., a trigger function) configured to output a signal (e.g., send a tag) to the user terminal 102 to output information indicating that the treatment is inappropriate when an event occurs in which the position of the tool overlaps with an area defined based on the characteristic portion data. The NG area data generation module 215 may be configured to generate check area data including the characteristic portion data and second characteristic portion data (an example of second NG area data) indicating a second area surrounding the area specified by the characteristic portion data with a predetermined dimension, and to generate check information for outputting information indicating that the treatment is inappropriate when the position of the tool overlaps with an area defined by this check area data.

[0111] In step S950, the treatment support data output module 217 outputs the treatment support data. FIG. 14 is an example of a treatment support data output flow 1400 corresponding to step S950. The treatment support data output module 217 acquires, for example, AR data (base data) in step S1410. FIG. 20 is an example of an AR model 2000 of the mandible. The treatment support data output module 217 acquires, for example, feature part data in step S1420. FIG. 21 is an example of an image in which a mandible model 2110 (AR model) and a NG area 2160 of the mandibular canal are superimposed. The treatment support data output module 217 outputs the base data and the feature part data as treatment support data in step S1430. The treatment support data output module 217 outputs the treatment support data as a combination of the base data and the feature part data, for example, as shown in FIG. 21. The treatment support data output module 217 can store the treatment support data as training data 600 in the auxiliary storage device 202, for example.

[0112] The treatment support data may include the above check information. Here, the treatment support data may have a data structure including: feature portion data, which is 3D data related to a feature portion identified based on image processing of base data, which is 3D data related to a tissue identified based on detection amount characteristics in medical image data of a living body; and check information for outputting information indicating that the treatment is inappropriate when the position of the tool based on position information of the tool used for the treatment and the area defined by the feature portion data overlap. The treatment support data may also have a data structure including: base data, which is 3D data related to a tissue identified based on detection amount characteristics in medical image data of a living body; feature portion data, which is 3D data related to a feature portion identified based on image processing of the base data; and check information for outputting information indicating that the treatment is inappropriate when the position of the tool based on position information of the tool used for the treatment and the area defined by the feature portion data overlap. With this configuration, the evaluation module 315 in the user terminal 102 that has acquired the treatment support data can be made to output information indicating that the treatment is inappropriate when a predetermined event occurs in which the position of the tool overlaps with the area defined based on the feature portion data.

[0113] In step S960, the dentition model generating module 214 generates dentition model data. The dentition model generating module 214 generates dentition model data based on, for example, base data. The dentition model generating module 214 also generates dentition model data based on, for example, one or more of the base data and the characteristic part data. The dentition model generating module 214 generates dentition model data in, for example, an STL format. Here, for example, the dentition model data generated based on the base data and the characteristic part data generated based on the same base data are associated with each other. In addition, for example, the dentition model data and the characteristic part data are associated with each other and stored in, for example, the auxiliary storage device 202. Note that, if the second generating module generates the base data in, for example, an STL format in step S920, step 960 can be omitted.

[0114] In step S970, the dentition model generating module 214 outputs the created dentition model data to the 3D printer 103. As a result, the 3D printer models the dentition model. The configuration, type, etc. of the 3D printer are not particularly limited, and for example, one or a combination of two or more of various 3D printers such as a liquid vat photopolymerization type, an inkjet type, and a powder lamination type can be used. Note that the material used to model the dentition model is not limited to resin, and may be resin, ceramic, metal, or a composite material thereof in part or in whole. In this way, a dentition model of a patient to be used for training can be obtained (see, for example, FIG. 24).

[0115] The dental model of this embodiment is composed of only the mandible and the left part of the lower dentition. For this dental model, for example, a softer resin can be used to integrally form the model artificial periosteum and the artificial gingiva. This makes it possible to realize a dental model having an outer shape corresponding to the user's jaw, and training can be performed under conditions closer to actual surgery. This dental model is, for example, sent to the user. This dental model is missing a first molar on the left side of the mandible. Using this dental model, the user can receive training on, for example, surgery to embed an implant at the position of the first molar. The material used to form the dental model is not limited to resin, and may be resin, ceramic, metal, or a composite material thereof in part or in whole.

[0116] Although not limited thereto, the dentition model generating module 214 may generate dentition model data by using, for example, any of the above-mentioned AR model data 500 and data in an image file format generated by the AR data generating module 213. For example, with respect to hard tissues, the dentition model generating module 214 may generate upper jaw model data from data of the upper jaw and upper dentition portion, and generate lower jaw model data from data of the mandible and lower dentition portion. The dentition model generating module 214 may generate the upper jaw model data and the lower jaw model data so as to include, for example, any one or a combination of two or more of the above-mentioned characteristic portion data.

[0117] Furthermore, the dentition model generating module 214 may generate dentition model data by using, for example, any of the above-mentioned characteristic part data generated by the AR data generating module 213. The dentition model generating module 214 may generate dentition model data by, for example, reflecting a part or all of the characteristic part data in the base data. Specifically, for example, the dentition model generating module 214 may generate upper jaw model data and / or lower jaw model data including at least one or more characteristic parts of a tooth part, a dental pulp part, a jawbone part, a nerve or artery and vein, a cyst, a tumor, and other lesions. The dentition model generating module 214 may be configured to, for example, reflect any characteristic part data in the dentition model data and not reflect other characteristic part data in the dentition model data in response to an instruction from a user. The dentition model generating module 214 typically forms dentition model data such that the characteristic part data and the base data fit together without overlapping with each other. This allows any characteristic part to be structurally reflected in the upper jaw model and / or the lower jaw model. For example, it is possible to create a dentition model having a structure that corresponds in part or in whole to the AR model displayed in the system. In addition, by using such dentition model data, it is possible to create these characteristic parts of the upper jaw model and / or the lower jaw model in a different color and / or material from the other parts.

[0118] Furthermore, the dentition model generating module 214 may generate upper jaw model data and / or lower jaw model data that does not include at least one of the above-mentioned characteristic parts in the upper jaw part and the lower jaw part, leaving the characteristic part empty. For example, when cancellous bone tissue does not affect the treatment, the dentition model generating module 214 may generate dentition model data that does not include cancellous bone tissue and leaves the cancellous bone tissue part empty. By using such dentition model data, it is possible to form an upper jaw model and / or a lower jaw model that does not include cancellous bone tissue, thereby making it possible to reduce the weight of the dentition model, save on the material for forming, and reduce the cost.

[0119] [Machine Learning] The second generation module 213B of the AR data generation module 213 may generate feature portion data using a learning device instead of generating feature portion data by the above image processing. The learning device used by the second generation module 213B may be, for example, a trained model trained to use medical image data as input data and image data related to a feature portion of a predetermined living body tissue as output data. The pre-trained model is not limited to this, but for example, a medical image segmentation model such as 3DU-Net, Swin Unet3D, or BTSwin-Unet can be suitably used.

[0120] The characteristic parts of biological tissues are closely related to, for example, the detection amount characteristics and three-dimensional structure in medical images. Therefore, such a learning device can be realized as a deep learning device using DNN (Deep Neural Network), as well as a learning device using unsupervised learning, a learning device using supervised learning, a learning device using reinforcement learning, etc.

[0121] [Training Preparation] The flow of the treatment training by the user will be described. The user prepares a dental model and prepares for implant insertion surgery into the dental model. For example, based on a treatment plan created in advance, the user prepares an implant body (which may be for practice) and surgical tools required for the implant insertion surgery. Once the preparations are complete, the user accesses a dental treatment support site on the web via the user terminal 102, for example, and selects a desired training from a training menu. Here, for example, the user can select training for implant insertion surgery using an AR model that corresponds to the dental model obtained by the user and is linked by a user ID.

[0122] When the user selects the desired treatment content for the dentition model, the reception module 312 of the user terminal 102 accepts the selection of the treatment content. The reception module 312 cooperates with the management module 311 of the management server 101, and transmits information about the training selected by the user (for example, the "drilling process (drilling, making a hole)" step in an implant insertion surgery) to the management server 101. The management module 311 of the management server 101 that receives the information about the training selected by the user transmits AR data and check data corresponding to the selected training to the user terminal 102.

[0123] Then, the data acquisition module 313 acquires AR model data and check data (including NG area data 800) corresponding to the selected training (for example, the "drilling process (implantation hole formation)" step in the implant insertion surgery). Then, the display module 314 displays the AR model and the NG area on a display (for example, AR glasses) based on the AR model data and the NG area data. The display module 314 can display the AR model and the NG area on the display as, for example, a transparent image.

[0124] Here, when the imaging function of the user terminal 102 (for example, the camera 306 provided in the head mounted display) is not operating, the display module 314 may be configured to activate the imaging function. The AR model data is implemented with, for example, an object tracking function. The display module 314 is also configured to be able to recognize the three-dimensional shape of the object being imaged. Therefore, when the user images the dentition model with the camera 306, the dentition model is displayed on the display and the three-dimensional shape of the dentition model is recognized.

[0125] This allows the display module 314 to display the AR model and / or the NG area (not shown) in association with the dental model being imaged as the starting point. That is, the display module 314 displays the AR model in association with the dental model so that the orientations and dimensions of the dental model (up / down, front / back, left / right) match the orientations and dimensions of the AR model and the NG area (common origin). As a result, the user can roughly grasp where the NG area is located on the dental model.

[0126] The AR model, the NG area, and the dental model are created based on the same 3DCT data. Therefore, for more accurate tracking, the AR model, the NG area data, and the dental model may each be provided with corresponding three or more markers. For example, the evaluation module 315 may be configured to superimpose the AR model, the NG area, and the dental model in a state where at least one of the occlusal plane and the Camper plane coincides. The occlusal plane can be defined as, for example, a plane including the midpoint (incisal point) between the mesial angles of the left and right mandibular central incisors and the distal buccal cusp tips of the left and right mandibular second molars. The Camper plane can be defined as, for example, a plane including the subala point and the left and right upper edge points of the tragus. In addition, markers may be provided for each part defining the occlusal plane and / or the Camper plane, or for any combination of other characteristic parts such as the incisive papilla, the maxillary tubercle, and the hard palate. This makes it possible to superimpose and track the AR model, NG area, and dental model while reducing discrepancies in position, tilt, size, etc.

[0127] [Treatment and its evaluation] The user can perform a treatment on the dental model while wearing, for example, AR glasses. When forming an implant bed, the user uses, for example, a cutting tool (drill, cutting bar) to form an implant cavity (prepared hole) for embedding an implant body at a predetermined position (here, the position of the first molar on the left side of the mandible). When forming the implant cavity, for example, the position, depth, implant insertion direction, etc. must be appropriately managed. The evaluation module 315 evaluates the user's treatment on the dental model based on the check data, and outputs (displays) the evaluation result on the display.

[0128] <Treatment evaluation 1> FIG. 15 is an example of a treatment evaluation process flow 1500. In step S1510, the evaluation module 315 acquires, for example, position information of a tool during treatment. The evaluation module 315 captures an image of the tool together with the dentition model by, for example, the camera 306, and recognizes and processes the 3D form by an image processing function. The evaluation module 315 thereby acquires, for example, relative position information of the tool with respect to the dentition model. The evaluation module 315 can acquire, for example, position information of a tool being drilled by a user during formation of an implantation cavity in real time.

[0129] In step S1520, the evaluation module 315 compares the tool position based on the tool position information with the NG area defined by the NG area data 800. Then, in step S1530, the evaluation module 315 determines whether the tool position and the NG area overlap. For example, when evaluating a procedure based on the NG area 2260 shown in Fig. 22, the evaluation module 315 determines whether the position (e.g., a group of coordinates) of the tool 2240 overlaps with the NG area 2260 (e.g., a group of coordinates). In other words, it evaluates whether the detailed checkpoint ID satisfies the check condition defined in "CP001-04", "the distance between the implant and the mandibular canal is approximately 3 mm or more."

[0130] If the tool position and the NG area do not overlap (No in step S1530), the evaluation module 315 outputs information indicating that the treatment is appropriate in step S1540. On the other hand, if the tool position and the NG area overlap (Yes in step S1530), the evaluation module 315 outputs information indicating that the treatment is inappropriate in step S1550. The evaluation module 315 evaluates the treatment in real time, for example, and outputs the result.

[0131] The method of outputting whether the treatment is appropriate or not is not particularly limited. The evaluation module 315 may output that the treatment is appropriate by, for example, lighting a green or blue light on the display, and may output that the treatment is inappropriate by, for example, lighting a red or yellow light on the display. The evaluation module 315 may also output that the treatment is appropriate by, for example, displaying a character or symbol indicating that the treatment is appropriate on the display, and may output that the treatment is inappropriate by, for example, displaying a character or symbol indicating that the treatment is inappropriate on the display.

[0132] Furthermore, the evaluation module 315 may, for example, indicate that the treatment is appropriate by not outputting anything when the treatment is appropriate, and only output that the treatment is inappropriate when the treatment is inappropriate. The evaluation module 315 may output information indicating that the treatment is appropriate or inappropriate, for example, by causing the user terminal 102 to execute one or more actions that the user can perceive. The actions that the user can perceive are, for example, one or a combination of two or more of the following: display of information using light, characters, figures, etc., generation of sound, generation of vibration, etc.

[0133] The evaluation module 315 may output that the treatment is appropriate, for example, by outputting a brisk and clear correct sound from the speaker, or may output that the treatment is inappropriate, for example, by outputting an incorrect sound such as a low cracking sound (for example, a buzzer sound). Further, the evaluation module 315 may output that the treatment is appropriate, for example, by causing the oscillator to output one vibration, or may output that the treatment is inappropriate, for example, by causing the oscillator to output a plurality of continuous vibrations. The evaluation module 315 may output any combination of the above.

[0134] For example, as shown in FIG. 7, one treatment step in training may include a plurality of checkpoints. Therefore, in step S1560, the evaluation module 315 checks, for example, for each treatment step, whether the evaluation of all checkpoints has been completed. If the evaluation of all checkpoints has not been completed (No in step S1560), the evaluation of the next checkpoint is performed as shown in step S1570. Then, when the evaluation of all checkpoints is completed, for example (Yes in step S1560), the evaluation module 315 ends the treatment evaluation process flow 1500.

[0135] <Non-display mode of the AR model> In the above description, the display module 314 displayed the AR model and the NG area on the display during the user's treatment. In this case, the user can perform the treatment while checking the NG area with the head-mounted display on. However, the display module 314 may be configured to be able to receive, for example, a user's selection to hide the AR model and the NG area. When the display module 314 receives the selection to hide the AR model and the NG area, it does not display the AR model and the NG area on the display.

[0136] <Image training using the AR model> In the above description, the display module 314 displays the AR model and the NG area on the display while the user is receiving treatment. In this case, the user can perform treatment while checking the NG area while wearing the head-mounted display. However, the display module 314 may display the AR model and the NG area on the display while the user is not receiving treatment. For example, the display module 314 may display the AR model and the NG area on the display when the user is not receiving treatment.

[0137] In this case, the AR model may include, for example, a treatment result model. The treatment result model may be, for example, a treatment result model (for example, a tool model in an ideal cutting state) that is created under the supervision of a skilled dentist and represents an ideal treatment state for each step of treatment. The display module 314 may also display this treatment result model (for example, a tool model) as an AR model. This allows the user to perform image training on the ideal treatment for the patient's dentition model, for example, before actual training or treatment. This may reduce the burden of actual training or treatment.

[0138] <Treatment evaluation 2> In the above description, the user performs the treatment while wearing the head mounted display. However, if the user does not like the treatment while wearing the head mounted display, the evaluation module 315 may evaluate the treatment with respect to the dental model after the treatment by the user (post-treatment evaluation mode).

[0139] The display module 314 displays, for example, the AR model 2200 (and the NG area 2260) corresponding to the step in question, overlaid on the dentition model after the treatment. The evaluation module 315 acquires position information about the tool 2240 used in the treatment (step S1510), and compares the position of the tool with the NG area (step S1520).

[0140] If the tool position does not overlap with the NG area (No in step S1530), the evaluation module 315 outputs that the treatment is appropriate (step S1540). If the tool position overlaps with the NG area (Yes in step S1530), the evaluation module 315 outputs that the treatment is inappropriate (step S1550). With this configuration, the treatment can also be evaluated.

[0141] <Treatment evaluation 3> Treatment evaluation 3 is a flow when multiple NG areas are set for one checkpoint. For example, the NG area data generation module 215 processes the characteristic part data related to the mandibular canal, creates a first NG area data for the mandibular canal and the area surrounding it with a thickness of 3 mm, and can also create a second NG area data for the mandibular canal and the area surrounding it with a thickness of 3.5 mm.

[0142] The data acquisition module 313 acquires AR model data corresponding to the selected training (for example, the "drilling process (hole making)" step) and check data including the first NG area data and the second NG area data.

[0143] For example, the evaluation module 315 first confirms that the treatment is not completed and acquires the position information of the tool. The evaluation module 315 can acquire, for example, the relative position information of the tool with respect to the dentition model by, for example, capturing an image of the tool with the camera 306 and recognizing the 3D form of the tool with an image processing function. For example, the evaluation module 315 acquires, in real time, the position information of the tool being moved by the user during the treatment.

[0144] The evaluation module 315 compares the tool position (e.g., a coordinate group) based on the tool position information with the second NG area (e.g., a coordinate group) defined by the NG area data 800. Then, the evaluation module 315 determines whether the tool position and the second NG area overlap. In other words, for example, in relation to the check condition defined by the detailed checkpoint ID "CP001-04" in FIG. 7, it evaluates whether the distance between the implant and the mandibular canal is approximately 3.5 mm or more.

[0145] If the position of the tool does not overlap with the second NG area, the evaluation module 315 returns to the treatment completion confirmation step (first time) and continues the treatment. On the other hand, if the position of the tool overlaps with the second NG area, the evaluation module 315 outputs information indicating that the treatment should be performed with caution. In other words, the evaluation module 315 notifies the user that the first NG area has become close.

[0146] The evaluation module 315 then confirms that the procedure is not complete and obtains the tool position information. The evaluation module 315 then compares the tool position (e.g., a set of coordinates) based on the tool position information with the first NG area (e.g., a set of coordinates) defined by the NG area data 800. The evaluation module 315 then determines whether the tool position and the first NG area overlap. In other words, for example, in relation to the check condition defined by the detailed checkpoint ID "CP001-04" in FIG. 7, it evaluates whether the distance between the implant and the mandibular canal is approximately 3 mm or more.

[0147] If the position of the tool does not overlap with the first NG area, the evaluation module 315 returns to the confirmation step of the completion of the treatment (for example, for the second or subsequent times) and continues the treatment. On the other hand, if the position of the tool overlaps with the first NG area, the evaluation module 315 outputs information indicating that the treatment is inappropriate.

[0148] Note that the evaluation module 315 can end the treatment evaluation step when it is confirmed in the first treatment completion confirmation step that the treatment is completed. For example, the evaluation module 315 can complete the treatment without outputting information indicating that attention should be paid to the treatment. Also, the evaluation module 315 can end the treatment evaluation step when it is confirmed in the second treatment completion confirmation step that the treatment is completed. For example, the evaluation module 315 can complete the treatment without outputting information indicating that the treatment is inappropriate after outputting information indicating that attention should be paid to the treatment.

[0149] According to the above configuration, for example, before the drilling reaches the first NG area in the drilling step, etc., it is possible to warn that the drilling is close to the first NG area. This allows users who are not accustomed to reading dental CT images or users who have little experience in implant insertion surgery to recognize that they are performing the procedure close to an inappropriate area without performing an inappropriate procedure (i.e., without accumulating failure experience), and to gain experience in the procedure.

[0150] In addition, for an implant surgery instrument (for example, a surgical tool), depending on the tip shape of the tool (cutting tip) used, there may be a portion where the cutting depth is deeper than the effective cutting length. According to the above configuration, for example, for a tool used for the first time, the treatment sensation remembered by the user can be associated with the actual treatment result, enabling more appropriate treatment.

[0151] In this example, the NG area data generation module 215 generates two NG areas, a first NG area and a second NG area, but the number of NG areas and the method of setting the areas are not limited to the above example. For example, the NG area data generation module 215 may generate a location other than the ideal treatment area as a third NG area. In this case, the evaluation module 315 can output information indicating that the tool position is outside the ideal treatment area when the tool position overlaps with the third NG area. This allows users who are not accustomed to reading dental CT images or users who have little experience in implant insertion surgery to perceive the extent to which the ideal treatment area extends, and further cultivate treatment experience.

[0152] <Treatment evaluation 4> The procedure evaluation 4 is a flow for evaluating whether the angle of the tool is appropriate at one checkpoint. In an implant insertion operation, for example, in order to stably fix the implant body, it is required to insert at least a part of the implant body into a relatively hard part near the side of the jawbone (buccolingual side) while securing a certain distance from the buccal and lingual sides. In addition, in order to avoid perforation of the maxillary sinus, chin, etc., the implant body may be intentionally inclined with respect to the mandible (inclined insertion). Although there is a certain degree of leeway (tolerance) in the appropriate insertion position and insertion angle of the implant, even if the entire implant is placed in the appropriate insertion position, if the axis of the implant is too far off from the optimal angle, it is not preferable from the viewpoints of occlusal function, fixation force, and aesthetics. The data acquisition module 313 evaluates whether the angle of the tool when forming the insertion hole is appropriate in the selected training (for example, the "drilling process (hole making)" step).

[0153] The evaluation module 315 acquires, for example, position information (for example, a coordinate group) of the tool. The evaluation module 315 can acquire the position information of the tool by, for example, capturing an image of the tool with the camera 306.

[0154] The evaluation module 315 acquires the axis information of the tool, for example, from the position information of the tool. The evaluation module 315 can acquire the axis information of the tool used by the user for the treatment, for example, based on a comparison between the 3D shape information of the tool stored in the check data 700 and the axis information. Alternatively, the evaluation module 315 may be configured to calculate the axis direction of the tool by three-dimensionally analyzing a coordinate group of the tool extending in the longitudinal direction, for example.

[0155] The evaluation module 315 acquires an angle related to the treatment position of the tool based on the axis information of the tool. The axis information of the tool can be, for example, vector information defined based on the position information of two different points on the axis. This makes it possible to grasp, for example, the relative positional relationship of the user's treatment tool with respect to the ideal treatment posture of the tool (the tool posture in the treatment plan).

[0156] The check data includes, for example, treatment angle data of a tool used for treatment on the dentition model. The treatment angle data indicates a range of appropriate angles (postures) of the tool with respect to the treatment position of the dentition model, and can be set based on a treatment plan. The evaluation module 315 compares the angle of the tool with the appropriate angle range. The evaluation module 315 evaluates whether the angle of the tool with respect to the treatment position is outside the appropriate angle range, and outputs that the treatment is appropriate when the angle of the tool is within the appropriate angle range. On the other hand, when the angle of the tool is outside the appropriate angle range, the evaluation module 315 outputs that the treatment is not appropriate. With this configuration, the angle of the tool in the treatment can be evaluated. The evaluation of the angle of the tool in the treatment can be evaluated in parallel with other checkpoints (e.g., the position of the tool).

[0157] <Treatment evaluation 5> In the above treatment evaluations 1 to 3, a NG area was set based on the characteristic part data related to the running positions of the nerves and blood vessels, etc., by providing a predetermined distance around them. Then, it was evaluated whether the position of the tool overlapped with this NG area. In treatment evaluation 5, the characteristic part data related to the running positions of the nerves and blood vessels, etc. were used as NG area data, and the running positions of the nerves and blood vessels, etc. were used as NG area data to evaluate the treatment.

[0158] The evaluation module 315, for example, acquires position information (for example, a group of coordinates) of the tool. The evaluation module 315 acquires tool perimeter information indicating a predetermined range surrounding the tool based on the position information of the tool, for example. The evaluation module 315 can set a range surrounding the tool with a thickness of 1.5 mm as the tool perimeter information for a tooth located next to the treatment position, for example. The evaluation module 315 can also set a range surrounding the tool with a thickness of 3 mm as the tool perimeter information for an adjacent implant, for example. Furthermore, the evaluation module 315 can set a range surrounding the tool with a thickness of 5 mm as the tool perimeter information for a mental foramen, for example.

[0159] The evaluation module 315, for example, compares the tool periphery defined by the tool periphery information with the NG area. The NG area is, for example, a tooth located next to the treatment position, an adjacent implant, and a mental foramen. The evaluation module 315 evaluates whether the tool periphery overlaps with the NG area, and outputs that the treatment is appropriate when the tool periphery does not overlap with the NG area. On the other hand, when the tool periphery overlaps with the NG area, the evaluation module 315 outputs that the treatment is not appropriate. With this configuration, the treatment (for example, position, depth, angle, etc.) can be evaluated. Note that this treatment evaluation can simultaneously evaluate multiple checkpoints (for example, a tooth located next to the treatment position, an adjacent implant, and a mental foramen, etc.).

[0160] <Treatment evaluation 6> The above procedure evaluations 1 to 5 explained the evaluation of the formation of one implant cavity using one tool. In implant insertion surgery, when forming an implant cavity, multiple tools with different cutting diameters are used in order of decreasing diameter to gradually enlarge the shape of the hole. In procedure evaluation 6, an evaluation is made on the case where a first tool and a second tool of different diameters are used to cut in the same position.

[0161] The user forms an implantation cavity using a first tool. The formation of the hole using the first tool can be evaluated by the above-mentioned treatment evaluation method. The evaluation module 315, for example, acquires position information (for example, a coordinate group) of the first tool. Then, the evaluation module 315 acquires position information of the first treatment area by the first tool. Here, the evaluation module 315 estimates the first treatment area as an overlapping portion between the jawbone part based on the position information of the jawbone part included in the AR model and the position of the tool based on the position information of the tool. Then, the evaluation module 315 generates position information of the first treatment area based on the overlapping portion between the jawbone part and the tool position.

[0162] The evaluation module 315 generates position information of the second NG area based on the generated position information of the first treatment area so that an area that is a predetermined distance or more away from the first treatment area becomes the second NG area. This predetermined distance that defines the second NG area corresponds to a value obtained by adding a radius difference between the first tool and the second tool to a deviation tolerance.

[0163] The user uses a second tool to form a hole overlapping the first treatment area. The evaluation module 315 acquires position information of the second treatment area by the second tool. Specifically, the evaluation module 315 first acquires position information of the second tool, and generates position information of the second treatment area in the same manner as the first treatment area. Then, the evaluation module 315 compares the second NG area with the second treatment area, and evaluates whether the second NG area and the second treatment area overlap.

[0164] When the second NG area and the second treatment area do not overlap, the evaluation module 315 outputs that the treatment is appropriate. On the other hand, when the second NG area and the second treatment area overlap, the evaluation module 315 outputs that the treatment is not appropriate. With this configuration, it is possible to evaluate whether the treatment is appropriate when the same position is treated in overlapping fashion. It is also possible to similarly evaluate when the same position is treated three or more times in overlapping fashion.

[0165] In this treatment evaluation 6, in an environment where the communication speed between the management server 101 and the user terminal 102 is sufficiently high, the generation of the location information of the second NG area by the evaluation module 315 may be executed by the NG area data generation module 215.

[0166] <Treatment evaluation 7> In procedure evaluation 6, the case where a first tool and a second tool of different diameters are used to cut at the same position is evaluated. When embedding multiple implants, for example, multiple implant holes may be formed at adjacent positions. In this case, adjacent implants must be spaced at least 3 mm apart, based on the distal or mesial cervical area. Therefore, in procedure evaluation 7, the case where implant holes are formed at adjacent positions is evaluated.

[0167] The user uses a tool to form a first hole. The formation of the first hole can be evaluated by treatment evaluations 1 to 5 or the like. The evaluation module 315 acquires, for example, position information of the tool (for example, a coordinate group). Then, the evaluation module 315 acquires position information of the formed first treatment area. The method of acquiring the position information of the treatment area can be the same as in treatment evaluation 6.

[0168] Then, the evaluation module 315 generates position information of the second NG area so as to include the first treatment area and an area of ​​a predetermined range surrounding the first treatment area. The predetermined distance defining the second NG area can be set in the treatment plan so that the distance between the cervical parts of the first implant and the adjacent second implant is 3 mm or more, taking into consideration the dimensions of the implant bodies and crown parts used in the first implant and the adjacent second implant.

[0169] The user continues to use the same tool, for example, to form a second hole at the formation position of the insertion hole of the second implant. The evaluation module 315 acquires position information of the second treatment area for the second hole. Specifically, the evaluation module 315 first acquires position information of the tool when the second hole is formed, and estimates the second treatment area as an overlapping portion between the jawbone part based on the position information of the jawbone part included in the AR model and the position of the tool based on the position information of the tool when the second hole is formed. Then, the evaluation module 315 generates position information of the second treatment area based on the overlapping portion between the jawbone part and the tool position.

[0170] The evaluation module 315 then compares the second NG area with the second treatment area and evaluates whether the second NG area and the second treatment area overlap. When the second NG area and the second treatment area do not overlap, the evaluation module 315 outputs that the treatment is appropriate, and when the second NG area and the second treatment area overlap, the evaluation module 315 outputs that the treatment is inappropriate. With this configuration, it is possible to evaluate whether the treatment is appropriate when implants are placed in adjacent positions. It is also possible to similarly evaluate when three implants are placed adjacent to each other.

[0171] In this treatment evaluation 7, in an environment where the communication speed between the management server 101 and the user terminal 102 is sufficiently high, the generation of the location information of the second NG area by the evaluation module 315 may be executed by the NG area data generation module 215.

[0172] <Control of the second prohibited area> With regard to treatment evaluations 6 and 7, the evaluation module 315 (which may be the NG area data generation module 215; the same applies below) may be configured to change the second NG area for the second treatment relative to the first NG area for the first treatment based on the check data when at least one of the tool and treatment position of the second treatment is different from that of the first treatment. Such processing can be suitably realized, for example, by flag processing.

[0173] For example, the evaluation module 315 can be configured in advance to execute a process of calculating information about the first treatment area and a predetermined range of areas surrounding the first treatment area based on the flag A. For example, if the tools used for the first treatment and the second treatment are the same, the flag A is set, and if the tools used for the first treatment and the second treatment are different, the flag A is not set. According to such a configuration, for example, as shown in treatment evaluation 6, if the tools used for the first treatment and the second treatment are different, the flag A is not set, so the evaluation module 315 operates not to include the above-mentioned predetermined range of areas in the second NG area. For example, as shown in treatment evaluation 7, if the tools used for the first treatment and the second treatment are the same, the flag A is set, so the evaluation module 315 operates to include the above-mentioned predetermined range of areas in the second NG area.

[0174] In other words, if the tools used for the first treatment and the second treatment are the same, the evaluation module 315 (an example of an evaluation unit) includes the first treatment area and a specified area surrounding the first treatment area in the second NG area, and if the tools used for the first treatment and the second treatment are different, the evaluation module 315 (an example of an evaluation unit) does not include the first treatment area and a specified area surrounding the first treatment area.

[0175] Also, for example, the evaluation module 315 can be configured in advance to execute a process of calculating information about an area farther away than a predetermined range surrounding the first treatment area based on a predetermined flag B. For example, when the treatment area is the same between the first treatment and the second treatment, the flag B is set, and when the treatment area is different between the first treatment and the second treatment, the flag B is not set. According to such a configuration, for example, as shown in treatment evaluation 6, when the treatment area is the same between the first treatment and the second treatment, the flag B is set, so the evaluation module 315 operates to include the area of ​​the above-mentioned predetermined range in the second NG area. And, for example, as shown in treatment evaluation 7, when the treatment area is different between the first treatment and the second treatment, the flag B is not set, so the evaluation module 315 operates not to include the area of ​​the above-mentioned predetermined range in the second NG area.

[0176] In other words, if the treatment area is the same for the first treatment and the second treatment, the evaluation module 315 (an example of an evaluation section) includes an area that is further than a specified range surrounding the first treatment area in the second NG area, and if the treatment area is different for the first treatment and the second treatment, the evaluation module 315 (an example of an evaluation section) does not include an area that is further than a specified range surrounding the first treatment area in the second NG area.

[0177] The above flags A and B may be used in combination. For example, if the tools used in the first and second treatments are the same, flag A is set, and if the tools used in the first and second treatments are different, flag B is set. As a result, for example, as shown in treatment evaluation 7, if the tools used in the first and second treatments are the same, flag A is set, and the evaluation module 315 operates to include the first treatment area and a predetermined range surrounding it in the second NG area. And, for example, as shown in treatment evaluation 6, if the tools used in the first and second treatments are different, flag B is set, and the evaluation module 315 operates to include an area farther away than the predetermined range surrounding the first treatment area in the second NG area.

[0178] In other words, if the tools used for the first treatment and the second treatment are the same, the evaluation module 315 (an example of an evaluation section) includes the first treatment area and a predetermined range surrounding the first treatment area in the second NG area, and if the tools used for the first treatment and the second treatment are different, the evaluation module 315 (an example of an evaluation section) includes an area further away than the predetermined range surrounding the first treatment area in the second NG area.

[0179] Also, for example, if the treatment area is the same between the first treatment and the second treatment, flag B is set, and if the treatment area is different between the first treatment and the second treatment, flag A is set. As a result, for example, as shown in treatment evaluation 6, if the treatment area is the same between the first treatment and the second treatment, flag B is set, so the evaluation module 315 operates to include an area farther away than a predetermined range surrounding the first treatment area in the second NG area. And, for example, as shown in treatment evaluation 7, if the treatment area is different between the first treatment and the second treatment, flag A is set, so the evaluation module 315 operates to include the first treatment area and a predetermined range of areas surrounding it in the second NG area.

[0180] In other words, if the treatment area is the same for the first treatment and the second treatment, the evaluation module 315 (an example of an evaluation section) includes an area farther away than a specified range surrounding the first treatment area in the second NG area, and if the treatment areas are different for the first treatment and the second treatment, the evaluation module 315 (an example of an evaluation section) includes the first treatment area and an area of ​​a specified range surrounding the first treatment area in the second NG area.

[0181] According to the above configuration, when the second NG area is calculated based on the first or second treatment area, the operation of the evaluation module 315 can be simplified.

[0182] <Treatment evaluation 8> When implanting an implant, a plurality of tools with different cutting diameters are used to form an implantation cavity. The tools may vary depending on, for example, the dimensions of the implant to be used and the hardness of the patient's mandible. For example, to form one implantation cavity, about 5 to 8 tools (cutting tips) are selected and used from about 6 to 10 tools. A user who is not familiar with using tools may mistake the tool to be used. Therefore, in the treatment evaluation 8, the type of cutting tools used to form the implantation cavity and the order of use are evaluated. The check data 700 includes information on the cutting tools used to form the implantation cavity and the order of use for each patient (i.e., for each AR model ID) based on the treatment plan.

[0183] The user mounts a cutting tool on the cutting drill to prepare for forming a first hole. For example, the user captures an image of the tool to be used with the camera 306. The evaluation module 315 acquires information about the tool to be used in the treatment, for example, based on the captured image data. The information about the tool is a feature for identifying the tool that is provided in advance on the tool, and may be, for example, the 2D or 3D shape of the tool, a color identifier, an identification code (letters, numbers, etc.), an engraving, a two-dimensional code, an RFID, etc.

[0184] The evaluation module 315, for example, compares the acquired information about the tool with information about the tool to be used for each treatment content. Then, the evaluation module 315 evaluates whether the acquired information about the tool corresponds to the information about the tool to be used based on the check data 700. For example, the evaluation module 315 checks whether the acquired information about the tool (e.g., a tool ID) matches the information about the tool to be used first recorded in the check data.

[0185] When the acquired information on the tool corresponds to the information on the tool to be used, the evaluation module 315 outputs that the tool is appropriate, and when the acquired information on the tool corresponds to the information on the tool to be used, the evaluation module 315 outputs that the tool is not appropriate. With this configuration, it is possible to evaluate whether the tool that the user is going to use for the treatment is appropriate. Note that the second and subsequent tools can also be evaluated in order, for example, for each treatment step.

[0186] <Treatment evaluation 9> 23 is another example of the screen 2300 during treatment evaluation, in which a guide 2330 is attached to a dental arch model. The guide 2330 is a treatment guide that can be used in surgery such as apicoectomy and radicular cystectomy. The guide 2330 is, for example, a clip type, and has a size that covers two adjacent teeth in the mesial and distal directions relative to the tooth loss position at the treatment position. Although such a clip type guide is easy to handle, it has a problem of poor attachment stability. Therefore, in treatment evaluation 9, the attachment of this clip type guide is evaluated.

[0187] This guide 2330 is designed so that one guide hole located on the buccal side of the maxilla 2320 when attached to the tooth 2310 corresponds to a position of a lesion such as a tumor at the root end of the tooth. This guide 2330 is designed to be able to appropriately guide the drill position (cutting position) to the lesion. The check data 700 includes guide model data corresponding to this clip-type guide 2330 for each patient (i.e., for each AR model ID) based on the treatment plan.

[0188] The user attaches the guide 2330 to be used for treatment to the dental model, and then captures the dental model with the guide attached by the camera 306. The evaluation module 315 then acquires position information of the guide with respect to the dental model. The position information of the guide 2330 can be acquired, for example, as three-dimensional coordinate group information with respect to the dental model. When the relationship between the position information of the guide 2330 and the position information of the dental model does not satisfy a predetermined standard based on the check data, the evaluation module 315 outputs information indicating that the method of attaching the guide 2330 to the dental model is not appropriate.

[0189] <Gap volume> For example, specifically, the evaluation module 315 acquires position information of the guide with respect to the dentition model, and calculates the volume of the space between the position (e.g., coordinate group) of the guide 2330 based on the position information of the guide and the position (e.g., coordinate group) of the dentition model based on the position information of the dentition model. This volume can be calculated based on spatial coordinates. Then, when this calculated volume is larger than a predetermined standard based on the check data (e.g., 10% or more of the volume of the part surrounded by the clip wall of the clip-type guide having a substantially U-shaped cross section), the evaluation module 315 can evaluate that the method of attaching the guide to the dentition model is not appropriate. Note that in FIG. 23, the clip-type guide 2330 has notches at two locations along the perspective direction, but when calculating the volume, it can be assumed that there are no notches and that the clip wall having a substantially U-shaped cross section is smoothly continuous.

[0190] <Tooth tip gap> Furthermore, the evaluation module 315, for example, acquires position information of the guide 2330 relative to the dental model, and compares the position of the guide based on the position information of the guide (for example, a group of coordinates) with the position of the dental model based on the position information of the dental model (for example, a group of coordinates). If there is a gap of a predetermined dimension or more (for example, 3 mm or more) between the tip of the tooth in the dental model and the inner surface of the clip wall of the guide 2330, it can be evaluated that the method of attaching the guide to the dental model is not appropriate.

[0191] The evaluation module 315 can extract the surface structures of the teeth and the guide 2330 by, for example, employing a surface rendering process based on the position information (coordinate group) of the teeth and the guide. The evaluation module 315 calculates the distance between the tip of the tooth (for example, the downward protruding portion) and the opposing surface of the guide 2330, and can evaluate that the method of attaching the guide to the dentition model is not appropriate if the distance is greater than a predetermined threshold value (for example, 3 mm).

[0192] <Tooth coverage rate> The evaluation module 315, for example, acquires position information of the guide relative to the dentition model, and compares the position of the guide based on the position information of the guide (for example, a set of coordinates) with the position of the dentition model based on the position information of the dentition model (for example, a set of coordinates). Then, for a tooth at least partially covered by the guide in the dentition model, if the surface coverage rate of the guide is less than a predetermined rate (for example, 80% or less), it can be evaluated that the method of attaching the guide to the dentition model is inappropriate. The evaluation module 315 can extract the surface structures of the tooth and the guide, for example, by employing a surface rendering process based on the position information (set of coordinates) of the tooth and the guide.

[0193] The evaluation module 315 calculates, for example, the ratio of the guide projection area on the tooth to the cheek tooth surface area when the buccal part of the guide is projected from the cheek side onto the tooth surface. At this time, the predetermined surface coverage ratio may be calculated, for example, by assuming that there is no notch for the notch part provided in the guide, and the guide projection area on the tooth may also be calculated assuming that there is no notch. Then, the evaluation module 315 can evaluate that the method of attaching the guide to the dentition model is inappropriate when the surface coverage ratio of the tooth by the guide does not meet a predetermined standard.

[0194] According to the above configuration, if the guide is improperly attached, the user can reattach the guide, thereby enabling the implant to be embedded in a more accurate position.

[0195] <Invagination into the maxillary sinus wall> If the maxillary sinus is located near the cutting position, the AR data generation module 213 can create feature data based on the maxillary sinus. The NG area data generation module 215 can create NG area data with the maxillary sinus and a part of a certain thickness surrounding it as the NG area based on the feature data related to the maxillary sinus. The evaluation module 315 can evaluate the procedure for forming the cutting hole based on the NG area data related to the maxillary sinus. The procedure evaluation method in this case can refer to, for example, one or more of the procedure evaluations described above as appropriate.

[0196] <Other treatment evaluations> Fig. 24 shows another example of the dental model. Fig. 25 shows an example of a support base for the dental model. In order for the display module 314 to more accurately superimpose the dental model and the AR model, for example, the dental model may be provided with a support base of a predetermined dimension that can be used as a reference for superimposition. This support base may also serve as a support base that supports an overhanging portion when forming the dental model with a 3D printer of the liquid vat photopolymerization type. The display module 314 can display the dental model and the AR model in a reliable correspondence with each other by using, for example, position information of three or more predetermined points of this support base as a reference.

[0197] The support base may be designed, for example, to be a mounting base for mounting the dental model in the oral cavity of a humanoid doll. The dimensions of such a support base can be designed to match the shape of the oral cavity of the humanoid doll so that the dental model can be mounted in the oral cavity of the humanoid doll in a more natural posture. This allows the dental model to be mounted in the oral cavity of the humanoid doll in a posture close to reality, and allows training in the treatment using the dental model under more realistic conditions.

[0198] In the above embodiment, the dentition model is that of a patient to be treated, but the dentition model is not limited to that of a patient to be treated. For example, the dentition model is not limited to that of a patient to be treated, but may be that of another living body, or may be a dentition model generated from 3D printer data of a typical case (including a virtual model case, etc.; the same applies below). In addition, other data, such as data of characteristic parts and dental CT data, is not limited to that specific to the patient. The other data may be, for example, that of another living body, or may be data of a typical case, such as data of characteristic parts of a typical case or dental CT data of a typical case.

[0199] In the above embodiment, the present technology has been described using dental treatment as an example, but the treatment on the living body is not limited to dental treatment, and can be, for example, any surgical operation on the whole body of a person without any particular limitation. In this case, various medical imaging data can be used instead of dental CT imaging data.

[0200] Each module disclosed in the above embodiment may be configured by combining multiple sub-modules. Furthermore, some or all of the operations performed or functions realized by one module may be performed or realized by another module.

[0201] In the above embodiment, one or more functional elements implemented by the execution of each module of the user terminal 102 may be implemented by the execution of a module of the management server 101 .

[0202] The present technology not only provides an invention related to the dental treatment support system 100, but also, in another aspect, can provide an invention related to a dental treatment support method.

[0203] The present technology also provides a program for making a computer execute each procedure in the dental treatment support method. The program may be composed of one program, or may be composed of two or more subprograms. The program may also be a program for making a computer execute any one or two or more of the above procedures.

[0204] Furthermore, one or more of the management servers 101 and one or more user terminals 102 constituting the dental treatment support system 100 may be installed in different countries. Furthermore, the management server 101 may be realized by one or more computers, any of which may be installed in different countries.

[0205] The present technology is not limited to the above-described embodiments, and various modifications are included. For example, the above-described embodiments have been described in detail to clearly explain the present technology, and are not necessarily limited to those having all of the configurations described. In addition, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. In addition, it is possible to add, delete, or replace a part of the configuration of each embodiment with another configuration.

[0206] In addition, the above-mentioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole by hardware, for example, by designing them as integrated circuits. In addition, the above-mentioned configurations, functions, etc. may be realized in software by a processor interpreting and executing a program that realizes each function. Information such as the program, table, file, etc. that realizes each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

[0207] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. In reality, it can be considered that almost all components are connected to each other. The above-mentioned embodiments disclose at least the configurations described in the claims. [Explanation of symbols]

[0208] 100...dental treatment support system, 101...management server, 211...user terminal management module, 212...dental CT acquisition module, 213...AR data generation module, 214...dental model generation module, 215...NG area data generation module, 216...server side evaluation module, 217...treatment support data output module, 311...management module, 312...reception module, 313...data acquisition module, 314...display module, 315...evaluation module

Claims

1. An acquisition unit that acquires medical image data of a living body; A first generation unit that generates base data, which is 3D data related to a tissue identified based on a detection amount characteristic in the medical image data; a second generation unit that generates feature portion data, which is 3D data related to a feature portion identified based on image processing of the base data; an output unit that outputs the base data and the feature portion data; A medical support system equipped with:

2. The system according to claim 1 , wherein the first generation unit generates the base data by performing processing on the organization according to a type of the organization.

3. The system according to claim 1 , wherein the second generating unit generates the characteristic portion data by performing a process on the characteristic portion according to a type of the characteristic portion.

4. The system according to claim 1 , wherein the first generation unit generates, as the base data, 3D data related to hard tissue identified by extracting information having a radiation absorption higher than a predetermined value based on the medical image data.

5. The system according to claim 4 , wherein the second generating unit identifies the characteristic portion of the living body by image processing of the base data based on at least one of the detection amount characteristics and a three-dimensional structure.

6. The system according to claim 4 , wherein the second generator is disposed within the hard tissue and identifies a region having a lower radiation absorption than the hard tissue as a hollow portion as the characteristic portion.

7. The system of claim 6 , wherein the second generator identifies the hollow portion, the elongated hollow portion, as a tube portion.

8. The first generation unit extracts a mandible and a tooth portion of the living body as the hard tissue, The system according to claim 7 , wherein the second generator extracts, as a mandibular canal, the canal portion of the mandible that extends in the anterior-posterior direction below the teeth.

9. The first generation unit extracts a mandible and a tooth portion of the living body as the hard tissue, The system according to claim 7 , wherein the second generating unit extracts the canal portion of the mandible extending in the anterior-posterior direction below the teeth as a running position of a nerve and / or a blood vessel.

10. The system according to claim 5 , wherein the second generation unit identifies a portion different from a main portion of the hard tissue as the characteristic portion based on a surface morphology identified by the image processing based on the three-dimensional structure.

11. The system according to claim 10 , wherein the second generator extracts, as the feature portion, a portion of the surface morphology where the degree of unevenness is high relative to a portion of the surface morphology where the degree of unevenness is low.

12. The system of claim 11 , wherein the second generator includes extracting the hole in the mandible, the hole being located below the teeth, as the mental foramen.

13. The system according to claim 11, wherein the second generating unit extracts the hole in the mandible, the hole being located 2 to 4 cm distal to the midline, as the mental foramen.

14. The system described in claim 11, wherein the second generation unit extracts as the mandibular foramen a hole in the mandibular bone that opens onto the lingual surface of the mandibular ramus and is positioned at the height of the occlusal plane (described in the specification) in the vertical direction of the mandibular ramus.

15. The first generating unit extracts a jawbone portion and a tooth portion of the living body as the hard tissue, The system of claim 4 , wherein the output unit outputs the base data corresponding to the jawbone and teeth, and the feature data corresponding to feature portions of the jawbone and teeth.

16. The system according to claim 1 , wherein the second generation unit extracts the feature portion using a trained model trained to use the medical image data as input data and image data related to the feature portion for a predetermined tissue of the living body as output data.

17. a check information generating unit that generates check information for outputting information indicating that the treatment is inappropriate when a position of a tool based on position information of the tool used for the treatment overlaps with an area defined based on the characteristic portion data; The system of claim 1 , comprising:

18. The characteristic portion data indicates an area not to be treated, The check information generation unit generating check area data including the characteristic portion data and second characteristic portion data indicating a second area surrounding the area specified by the characteristic portion data with a predetermined dimension; generating check information for outputting information indicating that the treatment is inappropriate when the position of the tool overlaps with an area defined by the check area data; 20. The system of claim 17.

19. A method for assisting medical care by a computer, comprising: Acquire medical image data of the living body; generating base data that is 3D data related to a tissue identified based on the detection amount characteristic in the medical image data; generating feature portion data, which is 3D data relating to a feature portion identified based on image processing of the base data; outputting the base data and the feature portion data; A medical support method, comprising:

20. A program for causing a computer to execute each step of the method according to claim 19.

21. feature portion data, which is 3D data related to a feature portion identified based on image processing of base data, which is 3D data related to tissue identified based on detection amount characteristics in medical image data of a living body; check information for outputting information indicating that the treatment is inappropriate when a position of a tool based on position information of the tool used for the treatment overlaps with an area defined by the characteristic portion data; A data structure including:

22. 22. The data structure of claim 21, further comprising 3D data relating to tissue identified based on detected volume characteristics in medical image data of a living body.

Images