Image processing device, image processing method and program

Abstract

The object is to facilitate the recognition of multiple biometric feature areas without omission. [Solution] The image processing device displays a biological feature area arrangement expanded image Eq on a display device 22, in which at least one of a plurality of annotation images Q is superimposed on a dentition expanded image Ep so that an annotation image showing each of a plurality of biological feature areas is located at a corresponding position among a plurality of biological feature area corresponding positions. An image for detailed observation at each of a plurality of three-dimensional positions is generated based on three-dimensional image data, a focus order of the plurality of biological feature areas is determined according to a predetermined rule, and in the display of the biological feature area arrangement expanded image, the annotation image changes to indicate that the biological feature areas are focused in the focus order, and the image for detailed observation according to the focus of the biological feature area is displayed on the display device.

Description

[Technical Field]

[0001] This disclosure relates to techniques for facilitating recognition of biometric feature regions. [Background technology]

[0002] Patent Document 1 discloses a technique for displaying a detected region of interest on a CT scan image.

[0003] Patent Document 2 discloses a technique for obtaining a panoramic tomographic image of the dentition using X-ray projection data of the dental and maxillofacial region obtained by X-ray CT imaging.

[0004] Patent Document 3 discloses a technology for detecting lesions by inputting CT images into a trained model. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Special Publication No. 2021-528751 [Patent Document 2] Japanese Patent Application Laid-Open No. 2008-229322 [Patent Document 3] Special Publication No. 2021-528751 Summary of the Invention [Problem to be solved by the invention]

[0006] According to the technology of displaying a detected region of interest on a CT scan image as in Patent Document 1, it may be time-consuming to search for a CT scan image that displays the region of interest. Also, when there are multiple regions of interest, it is required to make it easy to recognize all of the multiple regions of interest without missing any.

[0007] Therefore, an object of the present disclosure is to facilitate the recognition of multiple biometric feature regions without omission. [Means for solving the problem]

[0008] The image processing device processes image data obtained by CT scanning of the tissues forming the dentition to generate an image for diagnosis, and is equipped with a storage device that stores the image data, a processing device, and a display device that displays an image for observation based on the output of the processing device, and the processing device generates three-dimensional image data of the tissues forming the dentition based on the image data, detects a plurality of biological characteristic areas in the tissues forming the dentition based on the image data, specifies a plurality of three-dimensional positions where each of the plurality of biological characteristic areas exists in a coordinate system of the three-dimensional image data, generates an expanded image of the dentition in which the tissues forming the dentition are expanded based on the three-dimensional image data, specifies a plurality of biological characteristic area corresponding positions in the expanded image of the dentition that correspond to each of the plurality of three-dimensional positions, and the image processing device displays, on the display device, a biometric feature area arrangement expanded image in which at least one of the plurality of annotation images is superimposed on the dentition expanded image so that an annotation image showing each of the plurality of biometric feature areas is located at a corresponding position among the plurality of biometric feature area corresponding positions; generates detailed observation images at each of the plurality of three-dimensional positions based on the three-dimensional image data; determines a focus order of the plurality of biometric feature areas according to a predetermined rule; indicates, in the display of the biometric feature area arrangement expanded image, that the biometric feature areas are focused in the focus order by a change in the annotation image; and displays the detailed observation images according to the focus of the biometric feature areas on the display device.

[0009] The image processing method is an image processing method for generating an image for diagnosis by processing image data obtained by CT imaging of the tissues constituting the dentition, and includes generating three-dimensional image data of the tissues constituting the dentition based on the image data, detecting a plurality of biological characteristic areas in the tissues constituting the dentition based on the image data, specifying a plurality of three-dimensional positions where each of the plurality of biological characteristic areas exists in a coordinate system of the three-dimensional image data, generating an expanded image of the dentition in which the tissues constituting the dentition are expanded based on the three-dimensional image data, specifying a plurality of positions corresponding to the plurality of biological characteristic areas in the expanded image of the dentition, and generating an annotation image showing each of the plurality of biological characteristic areas based on the plurality of three-dimensional positions. and displaying, on a display device, a biometric feature area arrangement expanded image in which at least one of the plurality of annotation images is superimposed on the dentition expanded image so that at least one of the plurality of annotation images is positioned at a corresponding position among a number of biometric feature area corresponding positions, generating detailed observation images at each of the plurality of three-dimensional positions based on the three-dimensional image data, determining a focus order of the plurality of biometric feature areas in accordance with a predetermined rule, indicating that the biometric feature areas are focused in the focus order by a change in the annotation image in the display of the biometric feature area arrangement expanded image, and displaying the detailed observation images according to the focus of the biometric feature areas on the display device.

[0010] The program is a program for generating diagnostic images by processing image data obtained by CT scanning of the tissues constituting the dentition, and causes a computer to: generate three-dimensional image data of the tissues constituting the dentition based on the image data; detect a plurality of biological characteristic areas in the tissues constituting the dentition based on the image data; identify a plurality of three-dimensional positions in the coordinate system of the three-dimensional image data where each of the plurality of biological characteristic areas exists; generate an expanded image of the dentition in which the tissues constituting the dentition are expanded based on the three-dimensional image data; identify a plurality of positions in the expanded image of the dentition corresponding to the plurality of three-dimensional positions; the program executes a process of: displaying, on a display device, a biometric feature area arrangement expanded image in which at least one of the plurality of annotation images is superimposed on the dentition expanded image so that at least one of the plurality of annotation images is positioned at a corresponding position out of a number of biometric feature area corresponding positions; generating detailed observation images at each of the plurality of three-dimensional positions based on the three-dimensional image data; determining a focus order of the plurality of biometric feature areas in accordance with a predetermined rule; indicating, in the display of the biometric feature area arrangement expanded image, that the biometric feature areas are focused in the focus order by a change in the annotation image; and displaying, on the display device, the detailed observation images according to the focus of the biometric feature areas. [Effects of the Invention]

[0011] According to the present disclosure, it is possible to easily recognize multiple biometric feature regions without omission. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a schematic diagram showing an image processing device and a CT imaging device according to an embodiment. [Figure 2] FIG. 2 is a block diagram showing the electrical configuration of the image processing device and the CT imaging device. [Figure 3] FIG. 3 is a functional block diagram of the arithmetic circuit. [Figure 4] FIG. 4 is a flowchart showing an example of processing by the image processing device. [Figure 5] FIG. 5 is an explanatory diagram showing the FOV region. [Figure 6] FIG. 6 is an explanatory diagram showing an example of a region where the tissues constituting the dentition spread. [Figure 7] FIG. 7 is an explanatory diagram showing the positional relationship between the tissues constituting the dentition and the lesion area within the FOV region. [Figure 8] FIG. 8 is an explanatory diagram showing an example of coordinates of a lesion area in a tissue constituting the dentition. [Figure 9] FIG. 9 is an explanatory diagram showing an example of processing for developing tissues constituting the dentition into an expanded image of the dentition. [Figure 10] FIG. 10 is a diagram showing a developed image of the layout of biometric feature regions. [Figure 11] FIG. 11 is a flowchart showing a specific example of the process in step S7 of FIG. [Figure 12] FIG. 12 is a flowchart showing a specific example of the observation display process of FIG. [Figure 13] FIG. 13 is a diagram showing a display example of a detailed observation image and a biological feature region layout expanded image. [Figure 14] FIG. 14 is a flowchart showing an example of processing for the detailed observation image of FIG. [Figure 15] FIG. 15 is an explanatory diagram showing an example of a moving operation process for a detailed observation image. [Figure 16] FIG. 16 is an explanatory diagram showing an example of how annotation images change according to the focus order. [Figure 17] FIG. 17 is an explanatory diagram showing a modified example of the annotation image. [Figure 18] FIG. 18 is an explanatory diagram showing the focus order according to the first modified example. [Figure 19] FIG. 19 is a diagram showing a developed image of a biometric feature area layout according to the second modified example. [Figure 20] FIG. 20 is a diagram showing a developed image of a biometric feature area layout according to the third modified example. [Figure 21] FIG. 21 is an explanatory diagram showing a state in which the head is positioned by the head holder in the fourth modified example. [Figure 22] FIG. 22 is an explanatory diagram showing the positional relationship between the head holding part and the tissues constituting the dentition. [Figure 23] FIG. 23 is a schematic diagram showing an image processing device according to a fifth modified example. [Figure 24] FIG. 24 is a block diagram showing the electrical configuration of the image processing device. DETAILED DESCRIPTION OF THE INVENTION

[0013] {Embodiment} An image processing apparatus, an image processing method, and a program according to an embodiment will be described below.

[0014] <Configuration including image processing device and CT imaging device> FIG. 1 is a schematic diagram showing an image processing device 20 connected to a CT imaging device 10. As shown in FIG.

[0015] The CT imaging device 10 is a device that performs CT (Computed Tomography) imaging of the tissues that make up the dentition to obtain image data. The obtained image data includes data related to the tissues that make up the dentition. The image processing device 20 processes the image data to generate an image for diagnosis.

[0016] For example, the CT imaging device 10 includes an X-ray generator 11, an X-ray detector 12, a rotating arm 13, a support 14, and an imaging processing unit 15.

[0017] Three-dimensional coordinates are set for calculation purposes in the space in which the CT imaging device 10 exists. For example, the three-dimensional coordinates are set based on the orientation of the subject positioned for imaging, with the Z direction along the body axis, the X direction perpendicular to the Z direction and extending along the left-right direction of the subject, and the Y direction perpendicular to the Z direction and extending along the front-to-back direction of the subject. In this embodiment, the subject is positioned at the examination position in an upright position, so the Z direction is perpendicular to the floor surface, along which the support 14 extends.

[0018] The X-ray generator 11 includes an X-ray tube and is configured to emit an X-ray beam toward the subject. The X-ray detector 12 includes an X-ray detection sensor. The X-rays emitted from the X-ray generator 11 pass through the subject and are detected by the X-ray detector 12.

[0019] The rotating arm 13 is, for example, a member formed in a U-shape that opens downward. An X-ray generator 11 and an X-ray detector 12 are supported at both ends of the rotating arm 13 in an opposing state. A subject can be placed between the X-ray generator 11 and the X-ray detector 12. The subject is a part of the human body that includes tissues that form the dentition, i.e., the head including the jaw.

[0020] The support pillar 14 is erected so as to extend in the direction of gravity (vertical direction). A cantilever arm 14a is supported on the support pillar 14 so as to be movable up and down. A swivel arm 13 is rotatably supported on the cantilever arm 14a. The height position of the swivel arm 13 is adjusted along the support pillar 14 to match the height position of the head.

[0021] With the subject's head positioned between both ends of the rotary arm 13, the rotary arm 13 is rotated, causing the X-ray generator 11 and the X-ray detector 12 to rotate around the head, thereby performing X-ray CT imaging of the head and obtaining image data.

[0022] For example, when the rotating arm 13 rotates, X-ray imaging is performed at each small rotation angle. This allows X-ray projection image data (frame data) to be obtained for each small rotation angle. Three-dimensional volume data of the subject is generated based on a group of X-ray projection image data (frame data group) captured at different rotation angles. This three-dimensional volume data is three-dimensional image data that indicates the distribution of X-ray absorption rate of the subject in a three-dimensional coordinate system.

[0023] The CT imaging device 10 may include a head holder 9. The head holder 9 is a part that holds the head, which is the subject of imaging. The head holder 9 may include a chin rest 9a that supports the chin. The chin rest 9a can support the front and bottom of the chin of the head. This positions the head in a fixed position in the front-to-back and up-to-down directions. The head holder 9 may also include two side holders 9b that position the head from both sides. The two side holders 9b may be, for example, ear rods that contact both ears of the head. The two side holders 9b position the head in a fixed position in the left-to-right direction.

[0024] The image processing device 20 processes image data obtained by CT imaging to generate an image for diagnosis. The image data obtained by CT imaging may be a group of X-ray projection image data for each small rotation angle, or may be three-dimensional volume data. In this embodiment, an example will be described in which the image data obtained by CT imaging is a group of X-ray projection data for each small rotation angle.

[0025] The image processing device 20 comprises a processing unit 30, a display device 22, and user interfaces 24, 26.

[0026] The processing unit 30 is configured with a computer, a workstation, or the like. The processing unit 30 is connected to the imaging processing unit 15 of the CT imaging device 10 via a wired or wireless connection, and can transmit and receive various data to and from the imaging processing unit 15. In this embodiment, the processing unit 30 can receive image data obtained by CT imaging from the imaging processing unit 15. Some or all of the functions of the processing unit 30 may be realized by a cloud server.

[0027] The display device 22 is, for example, a liquid crystal display or an organic EL (electro-luminescence) display, and is connected by wire or wirelessly to the processing unit 30. The display device 22 can display an image for diagnosis based on the output for display of the processing unit 30.

[0028] The user interfaces 24, 26 are devices that accept instructions from a user of the image processing device 20. The user interface 24 may be, for example, a switch device such as a keyboard. The user interface 26 may be, for example, a pointer device such as a mouse. If the user interface is a switch device, the user's instructions can be input using the switch device alone. If the user interface is a pointer device, the user's instructions can be input by operating characters or images displayed on the display device 22. The user interface may be a touch panel.

[0029] <Electrical configuration of the image processing device and CT imaging device> FIG. 2 is a block diagram showing the electrical configuration of the image processing device 20 and the CT imaging device 10.

[0030] The CT imaging device 10 includes an imaging processing unit 15, an imaging unit driving mechanism 18, and a user interface 19.

[0031] The photographing processing unit 15 is composed of a computer including an arithmetic circuit 16 and a storage device 17 .

[0032] The arithmetic circuit 16 includes a processor 16a. The processor 16a may be a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit).

[0033] The storage device 17 is configured by a non-volatile storage device such as a flash memory or a hard disk drive. The storage device 17 may also be a memory circuit. The storage device 17 stores a program. The program describes the procedure for the CT imaging device 10 to perform CT imaging.

[0034] The imaging unit drive mechanism 18 is connected to the photographing processing unit 15. The imaging unit drive mechanism 18 includes a rotation drive mechanism for rotating the rotating arm 13. The rotation drive mechanism includes an actuator and a transmission mechanism. The actuator is an electric motor or the like that generates a rotation drive force. The transmission mechanism is gears, pulleys, or the like that transmit the rotation drive force of the actuator. The rotation drive force of the actuator is transmitted to the rotating arm 13 via the transmission mechanism, and the rotating arm 13 rotates at a timing and rotation speed according to a command from the photographing processing unit 15.

[0035] The user interface 19 is an interface for giving instructions to the CT imaging apparatus 10, and is a switch device, a pointer device, or the like. The user interface 19 is connected to the imaging processing unit 15. A user can give various instructions to the CT imaging apparatus 10 through the user interface 19.

[0036] The X-ray generator 11 and the X-ray detector 12 are also connected to the imaging processing unit 15. The X-ray generator 11 irradiates X-rays at the timing and output according to instructions from the imaging processing unit 15, and the X-ray detector 12 outputs the detection result to the imaging processing unit 15.

[0037] The processor 16a executes processing according to the program in the storage device 17, whereby the imaging unit drive mechanism 18 controls the rotation of the rotation arm 13 and also controls the X-ray irradiation operation of the X-ray generator 11. For each minute rotation angle of the rotation arm 13, the detection results of the X-ray detector 12 are input to the imaging processing unit 15, and the processor 16a generates X-ray projection image data for each minute rotation angle based on the detection results. The data 17a is stored in the storage device 17.

[0038] The image processing device 20 comprises a processing unit 30, a display device 22, and user interfaces 24, 26.

[0039] The processing unit 30 is connected to the display device 22 and the user interfaces 24, 26. The processing unit 30 can receive instructions from a user via the user interfaces 24, 26. The processing unit 30 can control the display of the display device 22.

[0040] The processing unit 30 is configured by a computer having an arithmetic circuit 32 as a processing device and a storage device 34 .

[0041] The arithmetic circuit 32 includes a processor 32a. The processor 32a may be a central processing unit (CPU). The processor 32a may include a graphics processing unit (GPU) or a processor for artificial intelligence (AI).

[0042] The storage device 34 is configured by a nonvolatile storage device such as a flash memory or a hard disk drive, or may be a memory circuit. The storage device 34 stores a program 34a and data 34b.

[0043] The program 34a describes a procedure for the image processing device 20 to process image data obtained by CT imaging and generate an image for diagnosis.

[0044] The data 34b includes image data transmitted from the CT imaging device 10 and data generated by the processing for generating the diagnostic image. After processing, the data 34b may be left as history data or may be deleted.

[0045] The processing unit 30 includes a connection port 36. The processing unit 30 is connected to the CT imaging device 10 through the connection port 36. The connection port 36 may include a terminal connected to a signal line of a wired cable extending from the CT imaging device 10 and a circuit for communication processing. The processing unit 30 and the CT imaging device 10 may be connected wirelessly, in which case the connection port 36 may include a circuit for wireless processing. The data 17a of the X-ray projection image data group from the CT imaging device 10 is transmitted to the processing unit 30 via the connection port 36 and stored in the storage device 34.

[0046] The arithmetic circuitry 32 reads the program 34a stored in the storage device 34 and executes the processes described in the program 34a, thereby enabling the arithmetic circuitry 32 to execute various processes for generating images for diagnosis, as will be described later. The processing unit 30 may also control CT imaging by the CT imaging device 10.

[0047] 3, the processing function unit realized by the arithmetic circuit 32 reading the program 34a may include a biological data processing unit 33. The biological data processing unit 33 may include a biological tissue data processing unit 33a and a biological feature region data processing unit 33b.

[0048] The biological tissue data processing unit 33a performs processing to generate three-dimensional data of the tissues constituting the dentition based on image data. The tissues constituting the dentition are, for example, tissues including the dentition and alveolar bone. The tissues constituting the dentition may be tissues including the dentition and jawbone. The jawbone in the tissues constituting the dentition may be the region of the jawbone that supports the teeth, and may include not only the alveolar bone but also its surrounding region. In this application, the alveolar bone is understood to be a partial region of the jawbone, that is, the part of the jawbone that forms the alveolus and supports the teeth. This understanding is in accordance with the explanation in the Encyclopedia of Dentistry. The tissues constituting the dentition may be horseshoe-shaped in a planar view. Note that the planar view may be considered as a planar view with the line of sight extending from the head to the feet in the axial direction of the body axis. The tissues constituting the dentition may extend to have a thickness in the buccolingual direction. The tissues constituting the dentition may be tissues that have at least the thickness of the dentition in the buccolingual direction. The tissues constituting the dentition may be tissues that have a thickness in the region of the jawbone that supports the dentition. The tissues constituting the dentition may include upper and lower dental arches and upper and lower jawbones supporting the upper and lower dental arches. Each of the upper and lower dental arches includes a plurality of teeth arranged in an arch shape. The upper jawbone has an alveolar bone supporting the teeth of the upper dental arch. The lower jawbone has an alveolar bone supporting the teeth of the lower dental arch. The biological tissue data processing unit 33a, for example, identifies the regions of each of the plurality of teeth and the regions of the upper and lower jawbones in the 3D volume data. The identification of the regions of the teeth and the upper and lower jawbones may be performed by applying a trained machine learning model. For example, a large amount of data in which the regions of the teeth and the regions of the upper and lower jawbones are mapped onto the 3D volume data is prepared as training data. Using the training data, a machine learning model trained to segment the regions of the teeth and the regions of the upper and lower jawbones in the 3D volume data is prepared. The machine learning model may be, for example, a model trained based on a semantic segmentation algorithm. The trained machine learning model is applied to the 3D volume data to identify the regions of each tooth and the upper and lower jaw bones in the 3D coordinate system, thereby distinguishing the tooth row region from the alveolar bone region based on the 3D volume data.

[0049] The regions of the plurality of teeth and the regions of the upper and lower jaw bones in the three-dimensional volume data may be identified by a region extraction process according to a predetermined rule, such as a pattern matching process.

[0050] The processing performed by the biological feature region data processor 33b is processing for detecting characteristic regions in the tissues constituting the dentition based on image data. Characteristic regions in the tissues constituting the dentition are, for example, regions in the tissues constituting the dentition where lesions have occurred. Lesions are changes caused by disease. Regions in which lesions have occurred exhibit a distribution of X-ray attenuation that differs from the distribution of X-ray attenuation exhibited by normal tissues constituting the dentition. Examples of diseases include chronic suppurative apical periodontitis and root granuloma. In the case of apical periodontitis, regions in the periphery of the tip of the tooth root exhibit lower X-ray attenuation than the normal state and its surroundings. Therefore, apical periodontitis can be detected when a low X-ray attenuation region is widespread around the periphery of the tooth root. For other lesions, regions in which lesions have occurred can be detected based on three-dimensional volume data obtained by CT imaging, based on the position of the tooth or jawbone in the tissues constituting the dentition, the spread pattern or distribution pattern of the X-ray attenuation region, and the like.

[0051] The detection by the biological feature area data processing unit 33b may be performed by applying a trained machine learning model similar to that used to identify the tissues constituting the dentition. For example, a large amount of data in which lesion areas are mapped onto 3D volume data is prepared as training data. Using the training data, a machine learning model trained to segment the lesion areas in the 3D volume data is prepared. The trained machine learning model is applied to the 3D volume data to detect the lesion area in a 3D coordinate system.

[0052] The detection of the lesion area in the three-dimensional volume data may be performed by image extraction processing, for example, pattern matching processing.

[0053] The detection of the lesion area does not have to be performed based on the three-dimensional volume data. For example, the lesion area may be detected by applying a machine learning model or pattern matching based on slice data obtained by slicing the three-dimensional volume data in any direction.

[0054] <Example of processing by processing unit> <Example of data processing for display> An example of data processing for display by the processing unit 30 will be described with reference to the flowchart of FIG.

[0055] After starting the process, in step S1, the processing unit 30 acquires captured image data from the CT imaging device 10. The captured image data is, for example, data of a field of view (FOV) region including the dental arch Ht and jawbone Hj of the head H (see FIG. 5). The FOV may be set to, for example, a cylindrical region, an elliptical cylindrical region, or a triangular prism region.

[0056] In the next step S2, the processing unit 30 generates three-dimensional volume data based on the captured image data.

[0057] In this embodiment, in step S1, the captured image data acquired from the CT imaging device 10 is a group of X-ray projection image data. In step S2, three-dimensional volume data is generated based on the group of X-ray projection image data. The three-dimensional volume data may be generated based on the group of X-ray projection image data in the CT imaging device 10. In this case, the processing unit 30 may acquire the three-dimensional volume data as image data.

[0058] It is also possible that the CT imaging device 10 generates multiple tomographic images based on a group of X-ray projection image data. In this case, the processing unit 30 may acquire multiple tomographic images as image data from the CT imaging device 10 and generate three-dimensional volume data by superimposing the multiple tomographic images.

[0059] In the next step S3, the processing unit 30 executes a process of generating three-dimensional image data of the tissues that constitute the dentition based on the image data, and a process of detecting a biological characteristic region L in the tissues that constitute the dentition based on the image data.

[0060] As shown in FIG. 6, the area Ee in which the tissues constituting the dentition E extend includes the dental arch Ht in which multiple teeth T are arranged, and the upper and lower jawbones Hj, as described above. The tissues constituting the dentition may also include the temporomandibular joint Hjo. In the three-dimensional volume data, the area Ee in which the tissues constituting the dentition E extend may be called the tissues constituting the dentition area Ee. The tissues constituting the dentition area Ee is a horseshoe-shaped area that is set by calculation, and it need only be an area that matches the shape and position of the tissues constituting the dentition. The shape of the tissues constituting the dentition area Ee may match the shape of the tissues constituting the dentition E of a standard skeleton, and when the shape of the tissues constituting the dentition E of an individual is known, it may be set to a shape that matches that individual.

[0061] As described above, the process of generating three-dimensional image data of the tissues that make up the dentition based on image data may also be a process of identifying each region of a plurality of teeth and each region of the upper and lower jaw bones based on three-dimensional volume data, and combining these regions to generate three-dimensional data of the tissues that make up the dentition.

[0062] Each region of the multiple teeth and each region of the upper and lower jawbones may be identified by the surface of the teeth and jawbones, i.e., the boundaries between the interior and exterior of the teeth and jawbones. The planes formed by the boundaries may be considered surfaces. In other words, each region of the multiple teeth and each region of the upper and lower jawbones may be identified as the surface shapes of each tooth and jawbone in the three-dimensional coordinate system of the three-dimensional volume data. Functional tissues that constitute a living body, such as teeth and jawbones, may be referred to as functional tissues that constitute the dentition of a living body. For example, teeth are functional tissues that constitute the dentition, with the function of chewing food, and jawbones are functional tissues that support the teeth. The tissues that constitute the dentition may be considered to include multiple functional tissues that constitute the dentition. Because the region of the jawbone boundary facing the teeth is important, a learning model including segmentation of the alveolar inner wall may be prepared to enable highly accurate detection of the alveolar inner wall.

[0063] Furthermore, the process of detecting a biometric feature region in the tissues that form the dentition based on image data may be a process of detecting a characteristic region in the tissues that form the dentition based on three-dimensional volume data, as described above. In this embodiment, an example is described in which the biometric feature region is a region where a lesion has occurred. Here, if the biometric feature region is, for example, a plurality of lesion regions, the collection of biometric feature regions is naturally composed of a collection of individual lesion regions. An individual biometric feature region that constitutes a collection of biometric feature regions, such as this individual lesion region, may be called a unitary biometric feature region. A region where a plurality of unitary biometric feature regions are collected may be called a collective biometric feature region. An image of a biometric feature region may be called a biometric feature region image.

[0064] Step S3 generates three-dimensional image data of the dental arch-forming tissues E, including the dental arch and jawbone, within the FOV, and detects the presence of a lesion area L (biological characteristic area L) in the dental arch-forming tissues E (see Figure 7).

[0065] The processing unit 30 can detect multiple biometric feature regions. In other words, when the processing unit 30 detects one biometric feature region (unitary biometric feature region), it does not end the detection of the biometric feature region but continues to detect other biometric feature regions. In this way, if multiple biometric feature regions exist in the dentition-forming structures E, the multiple biometric feature regions (collective biometric feature region) are detected.

[0066] In this embodiment, the explanation is given on the assumption that some biometric feature region is present. If no biometric feature region is detected, the processing may end. Furthermore, if only one biometric feature region is detected, the process of changing the focus, which will be described later, may be omitted.

[0067] In step S4, the processing unit 30 calculates the three-dimensional position, i.e., the three-dimensional coordinates, of the detected biometric characteristic area in the coordinate system of the three-dimensional image data of the dentition-forming tissues E. The three-dimensional coordinates of the three-dimensional image data are calculated based on the spatial coordinates of the CT imaging device, and for example, the X1 direction, which is the same as the X direction, the Y1 direction, which is the same as the Y direction, and the Z1 direction, which is the same as the Z direction, are set based on the orientation of the subject at the time of imaging. The three-dimensional position where the biometric characteristic area exists may also be called the three-dimensional position of the biometric characteristic area.

[0068] The coordinates of the biometric feature area are positions to be displayed as the location of the biometric feature area in the unfolded image described below. The coordinates of the biometric feature area may be the coordinates of a point located within the boundary of the lesion area detected in step S4. The coordinates of the biometric feature area may be, for example, the geometric center of the surface of the lesion area detected in step S3 (see FIG. 8). The coordinates may also be any position on the surface of the lesion area (for example, the lower or upper end).

[0069] In the next step S5, the processing unit 30 determines whether or not a command to display an unfolded image has been received. The command to display an unfolded image is received via the user interfaces 24, 26. When the user inputs a command to display an unfolded image via the user interfaces 24, 26, the processing unit 30 determines that a command to display an unfolded image has been received, and proceeds to step S6. When the user does not input a command to display an unfolded image, the processing unit 30 determines that a command to display an unfolded image has not been received, and proceeds to step S8.

[0070] In step S8, the processing unit 30 determines whether or not there is another command via the user interface 24, 26, etc. If it is determined that there is another command, the processing unit 30 executes the other process. If it is determined that there is no other command, the process returns to step S5, and the processing unit 30 repeats the process of determining whether or not there is a command to display the unfolded image.

[0071] When the process proceeds to step S6, the processing unit 30 generates a dentition expanded image Ep by expanding the dentition-forming tissues E based on the three-dimensional image data of the dentition-forming tissues E. The dentition expanded image Ep is an image in which the dentition-forming tissues E, which are arch-shaped or horseshoe-shaped in a planar view, are expanded to form a straight line in a planar view (see FIG. 9). For example, the dentition expanded image Ep is generated as follows: At each coordinate position in the vertical direction, a curve forming an arch or horseshoe shape that passes through the center of the dentition-forming tissues E in the buccolingual direction is calculated. At each position along the curve, the presence or absence of an image or the transparency in a direction perpendicular to the tangent to the curve is calculated. The presence or absence of an image or the transparency at each coordinate on the curve is expressed as the presence or absence of an image or the transparency at each coordinate on the straight line. The dentition expanded image Ep is generated by performing the above process at each coordinate position in the vertical direction and overlapping them in the vertical direction.

[0072] In addition, when the three-dimensional image data of the dental arch-forming tissues E is represented by semi-transparent data of the surface of the dental arch-forming tissues E, the areas that are the boundaries of the dental arch-forming tissues in the dental arch expanded image Ep are represented with low transparency and in dark colors.

[0073] The dentition expansion image Ep is a type of panoramic image in which the dental arch is expanded in a plane. The dentition expansion image Ep may also include the alveolar bone and jawbone. It may also include the jawbone. The dentition expansion image Ep may be an image of the dentition-constituting tissues E transmitted through the thickness direction, or may be a tomographic image of an arbitrary position in the thickness direction.

[0074] The dentition expanded image Ep may be an image obtained by viewing the entire area of ​​the tissues constituting the dentition E in a normal view. An image obtained by viewing the entire area of ​​the tissues constituting the dentition E in a normal view is an image obtained by viewing the entire tissues constituting the dentition E from in front of the midline of the head H (from a normal viewing direction Dv that views the anterior tooth region normal) when the dentition expanded image Ep is expanded in a direction perpendicular to the midline Md of the head H, as shown in Fig. 9.

[0075] The dentition expanded image Ep may be superimposed not only with 3D image data of the dentition-forming structures E but also with data obtained by converting 3D volume data into a panoramic image. The dentition expanded image Ep does not necessarily have to be expanded to form a straight line in a planar view, and may include curvature or refraction. In other words, the dentition expanded image Ep may be an image that allows the entire dentition-forming structures E to be observed at a glance. For example, the expansion may be processed so that the left and right end regions of the dentition-forming structures region Ee are displaced more anteriorly than the central region. A linear or nearly linear expansion in a planar view is referred to as "flat expansion," and the dentition expanded image Ep may be an image that is flatly expanded. The image process for generating the dentition expanded image Ep so that the entire dentition-forming structures E can be observed at a glance may be called an expansion process. In particular, a flat expansion process may be called a flat expansion process. A flat expansion may also be called a flat expansion.

[0076] The above process is similarly applied to the biometric feature region to calculate a biometric feature region corresponding position P2 in the dentition expansion image Ep. The biometric feature region corresponding position P2 may be considered to be a position obtained by converting the three-dimensional position P1 of the biometric feature region calculated in step S4 using the same geometric conversion process as that used to expand the dentition-constituting structures E into the dentition expansion image Ep.

[0077] The three-dimensional position P1 can be determined, for example, at the center of the biometric feature region. The center of the biometric feature region may be the geometric center or the center of gravity. The three-dimensional position P1 may be the position of a point (i.e., a position indicating a specific point) as the calculation target, but if the biometric feature region has an extent, it may also be the position of at least a part of its region. For example, it may be a certain region around the center including the center. The three-dimensional position may also be the position of the entire biometric feature region. Therefore, the corresponding position P2 may also be the position of a point or a region as the calculation target.

[0078] The area set by calculation when the dentition tissue area Ee is expanded may be considered to be the dentition expanded area Ex. A flat expansion of the dentition tissue area Ee may also be called a flat expansion, and in particular, the flatly expanded dentition expanded area Ex may be considered to be the dentition flat expanded area Exp.

[0079] The expansion of the tissues forming the dentition E may be performed by an expansion process that expands the image data of the tissues forming the dentition area Ee so that it fits the shape of the dentition expanded area Ex. In the following discussion, it is assumed that the area occupied by the tissues forming the dentition E and the tissues forming the dentition area Ee match, and that the area occupied by the dentition expanded image Ep matches the dentition expanded area Ex.

[0080] The state of the dentition-constituting tissue region Ee before expansion may be called the pre-expansion curved state, and the state of the dentition expansion region Ex after expansion may be called the post-expansion view state. The expansion process is preferably performed so that the buccolingual direction BLD in the pre-expansion curved state corresponds to the normal view direction NVD in the post-expansion view state.

[0081] The normal viewing direction may be a direction perpendicular to or normal to the normal plane of the dentition spread area Ex, but a viewing direction based on another concept may also be set. For example, a virtual viewpoint Ey1 may be set in the normal viewing direction at the center of the dentition spread area Ex, and image processing may be performed along the viewing direction EVD from this viewpoint.

[0082] In the dentition spread area Ex and / or the dentition spread image Ep, the area at the biometric feature area corresponding position P2 and corresponding to the biometric feature area L may be referred to as a biometric feature area corresponding area Lc.

[0083] Since the dentition tissue region Ee has a thickness in the buccolingual direction, the dentition expanded region Ex also has a thickness in the normal viewing direction corresponding to the thickness in the buccolingual direction. Furthermore, since the dentition tissue E has a thickness in the buccolingual direction as described above, the dentition expanded image Ep also has a thickness in the normal viewing direction corresponding to the buccolingual direction.

[0084] The dental arch expansion area Ex has a thickness corresponding to the dental arch tissue area Ee, and therefore, unlike a thin panoramic slice, it can be contained within the area even if there is a buccal-lingual bias in the location of the biological feature area (in image processing, the biological feature area image is contained within the area without blurring).

[0085] The coordinates of the three-dimensional position P1 may be calculated as coordinates in the FOV region, or as coordinates in the dentition-forming tissue region Ee.

[0086] In the expansion, a calculation may be performed to associate the three-dimensional position P1 of the biometric feature region with the biometric feature region corresponding position P2. The calculation to perform this correspondence may be called an original coordinate calculation. The original coordinate calculation may include a calculation to identify the position of the three-dimensional position P1 of the biometric feature region in the buccolingual direction. Also, a calculation may be performed to identify the position of the biometric feature region corresponding position P2 in the normal viewing direction corresponding to the buccolingual direction.

[0087] Then, as shown in FIG. 10, the processing unit 30 displays on the display device 22 a biometric feature region arrangement expanded image Eq in which the annotation image Q is superimposed on the row of teeth expanded image Ep.

[0088] For calculation purposes, three-dimensional coordinates may be set for the dentition spread area Ex. For example, the left-right extension direction of the dentition spread area Ex when unfolded is defined as the R direction. The direction parallel to the body axis is defined as the T direction. In this embodiment, the T direction, Z direction, and Z1 direction are the same direction. The direction parallel to the normal viewing direction is defined as the S direction. In the dentition spread image Ep, the left-right direction is defined as the R direction, and the up-down direction is defined as the T direction. Once the conditions for specifying the RT coordinate on the dentition spread image Ep and the S direction are determined, it becomes possible to mutually identify the X1Y1Z1 coordinate. Therefore, in areas where both the RT coordinate and the S coordinate are determined, such as the biometric feature area corresponding area Lc in the dentition spread area Ex, it is possible to mutually identify the X1Y1Z1 coordinate of the biometric feature area L. Coordinates on the three-dimensional image data, such as the X1Y1Z1 coordinates, may be referred to as three-dimensional image data coordinates, and three-dimensional coordinates may be referred to as three-dimensional image data three-dimensional coordinates. Coordinates on the dentition spread area Ex, like the RST coordinates, may be called dentition spread area coordinates, and the three-dimensional coordinates may be called dentition spread area three-dimensional coordinates.

[0089] The annotation image Q is an image showing a biometric feature area. The annotation image Q may be an image that exhibits a color, shape, or change that is distinguishable from the background expanded image of the arch of teeth Ep. For example, the annotation image Q may be an image that exhibits a color different from that of the expanded image of the arch of teeth Ep. More specifically, the annotation image Q may be an image that exhibits a conspicuous warm color, such as red, orange, or yellow. The annotation image Q may be an image that exhibits a change in display, such as blinking, a change in color, or a change in shape, that makes it distinguishable from the expanded image of the arch of teeth Ep. The annotation image Q may be an image that exhibits a shape that is distinguishable from the expanded image of the arch of teeth Ep, such as a circle, a regular polygon, a radial shape, a cross shape, or an exclamation mark shape.

[0090] The annotation image Q may have a shape that shows the outline of the lesion area L. In this case, the approximate shape and size of the lesion area L can be known by looking at the biological feature area layout expanded image Eq.

[0091] At least one annotation image Q is superimposed on the row of teeth extruded image Ep so as to be located at the biological feature region corresponding position P2 in the row of teeth extruded image Ep.

[0092] In the example shown in FIG. 10, the boundaries between the teeth and jawbone are indicated by lines in the dentition expansion image Ep. Furthermore, the annotation image Q is indicated by a double circle with a dashed line. In the example shown in FIG. 10, five annotation images Q are shown. In other words, if multiple biometric feature regions are detected, multiple annotation images Q corresponding to each of the multiple biometric feature regions are recognizably displayed in the biometric feature region arrangement expansion image Eq. The semi-transparent processing of the functional dentition-constituting tissues may be performed, for example, by forming the boundaries (which may be considered as the boundaries between existence and non-existence) of the functional dentition-constituting tissues with uniform, sparse dots, as in the tooth image THd shown in FIG. 10. This results in high transparency in the area Or perpendicular to the line of sight, and low transparency in the area Bd aligned with the line of sight, making the boundaries easily visible despite being semi-transparent. The dots may be non-uniform to the extent that they do not cause visual difficulties. As long as the boundary surfaces are transparent, image processing of a see-through surface, such as a mesh, may also be used. Whether it is dots or meshes, the transparency changes depending on the line of sight as the density varies. Image processing that changes the transparency depending on the line of sight may be called line of sight compatible transparency image processing.

[0093] An element whose transparency is increased, such as a dot that has become sparse compared to a dense state or a dot that has disappeared, is called a parent transparent element. In addition, increasing transparency is called parent transparency.

[0094] An element that reduces transparency, such as a dense dot compared to a sparse or non-existent state, is called an anti-transparency element. Also, reducing transparency is called anti-transparency.

[0095] The gaze direction compatible transparent image processing may be a combination of at least one of the following group A and at least one of the following group B. Group A: · Processing to enlarge or increase the parent transparency element of the normal boundary surface -Processing to promote parent transparency at the boundary of normal vision Treatment to reduce, diminish or eliminate the anti-transparency elements of the normal vision interface Treatment to suppress anti-transparency of the normal vision interface Group B: - Processing to reduce, decrease or eliminate parent transparency elements on the boundary surface of oblique or side views - Processing to suppress parent transparency at the boundary surface of oblique or side views Treatment to enlarge or increase the anti-transparency factor at the oblique or lateral viewing interface Treatment to promote anti-transparency at the interface of oblique or lateral vision The alveolar bone and jawbone portions may also be subjected to gaze direction-compatible transparent image processing.

[0096] In this embodiment, in order to explain the focus change process described later, the explanation is given on the assumption that there are multiple biometric feature regions. Note that even when there are multiple biometric feature regions, there may be cases where one or multiple annotation images Q corresponding to some of the multiple biometric feature regions are displayed.

[0097] If no biometric feature region is detected in step S3, the process may not end and the row of teeth expanded image Ep without the annotation image Q may be displayed.

[0098] Step S5 may be omitted, and after the processing of step S4 is completed, the processing of step S6 may be performed regardless of whether or not a command is issued.

[0099] After step S6, display processing for detailed observation is performed in step S7. When this display processing is completed, the processing of the processing unit 30 is completed.

[0100] <About focus processing> The processing in step S7 includes a process for generating a detailed observation image, a process for determining a focus order, and a process for focusing on the biometric feature regions in the focus order and displaying the detailed observation image on the display device 22. As a result, the biometric feature regions are focused in the focus order in the biometric feature region layout expanded image Eq, and a detailed observation image of the focused biometric feature region is displayed. Therefore, the user can observe the detailed observation image while recognizing the position of the focused biometric feature region by looking at the biometric feature region layout expanded image Eq. Because the biometric feature regions are focused in a predetermined focus order, it is easy to observe the biometric feature regions without omissions or overlaps. The focus order of the biometric feature regions may be the order of focusing from one unitary biometric feature region to another unitary biometric feature region. The focus order is determined for collective biometric feature regions of the same type, such as a "lesion" or a "root apex," and different focus targets will be different depending on the target of interest. Such biometric feature regions of the same type that are targets of interest may be called "biometric feature regions of interest."

[0101] The processing of step S7 will be explained in more detail with reference to the flowchart shown in FIG.

[0102] In step S11, the processing unit 30 determines a focus order. The focus order is determined according to a rule predetermined by the program 34a. The rule may be a rule that determines the focus order based on a plurality of biometric feature region corresponding positions P2 in the dentition expansion image Ep. For example, the rule may be defined as a rule that the biometric feature region corresponding positions P2 located on the upper dentition and jawbone are ranked before the biometric feature region corresponding positions P2 located on the lower dentition and jawbone, and a rule that the images are displayed in order from one side to the other in the left-right direction (for example, from left to right in front view). In this embodiment, the following description will be given assuming that the rule is applied.

[0103] The rule for determining the focus order may be other rules. For example, the rule may be defined simply as displaying from one side to the other in the left-right direction (for example, displaying from left to right in front view).

[0104] By determining the focus order, for example, focus order data is generated in which a plurality of biometric feature area corresponding positions P2 are stored in a predetermined array.

[0105] The processing unit 30 also determines an initial position. The initial position may be, for example, the first position in the determined focus order. For example, the biometric feature area corresponding position P2, which is located on the upper dentition and jawbone and is located at the leftmost position, may be set as the initial position. The feature area corresponding to the initial position may be called the initial feature area.

[0106] In the next step S12, annotation display processing is performed. The annotation display processing is processing for displaying an annotation image Q in the biological feature area arrangement expanded image Eq so that the position of the feature area displayed as the detailed observation image D can be recognized in the dentition expanded image Ep. In step S12, the detailed observation image D corresponding to the initial feature area is displayed. Therefore, for example, the annotation image Q corresponding to the initial feature area is displayed so as to be distinguished from the other annotation images Q. An example of displaying any one of the annotation images Q so as to be distinguishable from the other annotation images Q will be described in step S20.

[0107] In the next step S13, the characteristic area corresponding to the initial position is displayed for observation. The process of step S13 will now be described in more detail with reference to the flowchart of FIG.

[0108] In step S13, a process is executed to generate and display an image for detailed observation suitable for observation.

[0109] That is, in step S31, the three-dimensional coordinates of the initial feature region identified in step S11 are calculated. For example, in step S6, when a dentition-forming structure E is developed into a dentition-forming structure expanded image Ep based on the three-dimensional image data of the dentition-forming structure E to generate the dentition-forming structure E, the three-dimensional position P1 of the biological feature region is converted into a biological feature region corresponding position P2. This correspondence is stored in the storage device 34 as a table. The three-dimensional coordinates of the initial feature region are calculated by referring to the table. Alternatively, the three-dimensional coordinates of the initial feature region may be calculated by an inverse transformation process of the geometric transformation process that develops the dentition-forming structure E into the dentition-forming structure expanded image Ep. This identifies the position of the initial feature region in the three-dimensional image data of the dentition-forming structure E.

[0110] In the next step S32, a detailed observation image D is generated at a three-dimensional position of the three-dimensional image data based on the three-dimensional image data of the dentition-forming structures E. Then, the detailed observation image D is displayed on the display device 22 simultaneously with the biological feature region arrangement expanded image Eq (see FIG. 13). Note that the detailed observation images D at each of the multiple three-dimensional positions may be generated before display, or may be generated all at once when the multiple biological feature regions are detected.

[0111] The detailed observation image D is an image that allows for more detailed observation than the dentition expansion image Ep and is suitable for more detailed diagnosis. In this embodiment, an example will be described in which the detailed observation image D includes multiple cross-sectional images Da, Db, and Dc, more specifically, three cross-sectional images Da, Db, and Dc that are orthogonal to each other (see FIG. 13).

[0112] The three cross-sectional images Da, Db, and Dc are cross sections taken along mutually orthogonal planes that pass through the three-dimensional position P1 of the biological feature region. The three cross-sectional images Da, Db, and Dc are generated by identifying a plane that passes through the three-dimensional position P1 in the three-dimensional image data of the dentition-forming tissues E and determining the distribution of X-ray transmittance along that plane. The three cross-sectional images Da, Db, and Dc may represent the distribution of X-ray transmittance in a thick slice layer. The correspondence between the three-dimensional position P1 of the biological feature region and the biological feature region-corresponding position P2 is determined in the unfolding process, so it is also determined which coordinates of the three-dimensional position P1 correspond to in the FOV region and / or which coordinates of the dentition-forming tissue region Ee. The same applies to the coordinates of each point in the region shown in the dentition unfolded image Ep. Of course, the original coordinate calculation described above may be performed.

[0113] The three cross-sectional images Da, Db, and Dc may include a cross-sectional image Dc orthogonal to the buccal-lingual direction, a cross-sectional image Db orthogonal to the up-down direction, and a cross-sectional image Da orthogonal to both cross-sectional images Dc and Db. The cross-sectional image Dc orthogonal to the buccal-lingual direction may be a cross-section along the tangent direction of a curve along the extension direction of the tissues E constituting the dentition. The cross-sectional image Dc orthogonal to the buccal-lingual direction is an example of a cross-sectional image that views the biological characteristic area squarely. As the default cross-sections can be arbitrarily determined, the assignment of each cross-sectional image to each of the cross-sectional images Da, Db, and Dc can be changed as appropriate to the cross-sectional image orthogonal to the buccal-lingual direction, the cross-sectional image orthogonal to the up-down body axis direction, or the cross-sectional image orthogonal to both of these cross-sectional images.

[0114] The detailed observation image D may be, for example, a cross-sectional image with a higher resolution than the dentition expansion image Ep, or a transmission image. The detailed observation image D may be a three-dimensional image whose line of sight is changeable.

[0115] As a result, a detailed observation image D is generated at the three-dimensional position P1 of the biometric feature area corresponding to the annotation image Q selected through the user interfaces 24 and 26.

[0116] The display device 22 displays the image for detailed observation D simultaneously with the developed image for biological feature area arrangement Eq. At this time, the position of the image for detailed observation D relative to the developed image for biological feature area arrangement Eq is arbitrary. In Fig. 13, the developed image for biological feature area arrangement Eq and the image for detailed observation D are arranged side by side. The developed image for biological feature area arrangement Eq and the image for detailed observation D may also be arranged side by side vertically. The image for detailed observation D may also be displayed superimposed so as to partially enter the developed image for biological feature area arrangement Eq.

[0117] Furthermore, the layout of the cross-sectional images Da, Db, and Dc in the detailed observation image D is arbitrary. In Fig. 13, the rectangular display area of ​​the detailed observation image D is divided into two parts, vertically and horizontally. The cross-sectional image Dc orthogonal to the buccal-lingual direction is displayed in the lower right area, the cross-sectional image Db orthogonal to the body axis direction is displayed in the upper left area, and the cross-sectional image Da orthogonal to those is displayed in the lower left area.

[0118] Each of the three cross-sectional images Da, Db, and Dc may include an index Li indicating the position of the other cross-section. The index Li may be a straight line, a dashed line, a short line located in the central region of the cross-sectional images Da, Db, and Dc, or two marks located on the periphery of the cross-sectional images Da, Db, and Dc. In other words, the index Li may be any indication that can indicate the linear position of the other cross-section. In FIG. 11, each of the cross-sectional images Da, Db, and Dc includes a vertical line and a horizontal line as the index Li, which indicate the position of the other cross-section.

[0119] In the next step S33, display processing may be performed on the image for detailed observation D. After this display processing is completed, the processing of step S13 may end. Specifically, step S33 is a processing whose main contents are from step S40, which is the start step of processing related to an image adjustment operation for detailed observation, which will be described later, to step S43, which is the step of generating and displaying the image for detailed observation after adjustment.

[0120] The display process for the detailed observation image D may be, for example, a process of changing the cross-sectional positions of the cross-sectional images Da, Db, and Dc, as shown in the flowchart of FIG.

[0121] That is, after the display of the image D for detailed observation, the process proceeds to step S40, the process related to the operation to adjust the image for detailed observation is started, and step S41 is processed. In step S41, the processing unit 30 determines whether or not a movement operation has been performed. The movement operation is accepted, for example, by an operation on an index Li through the user interfaces 24, 26. For example, as shown in FIG. 15, by moving any of the indexes Li (denoted as index Lia in FIG. 15) with a pointer device such as a mouse, the cross-sectional position indicated by the index Lia is moved. In this way, the movement operation is accepted. The movement operation may be performed with a switch device such as a keyboard.

[0122] The movement operation in step S41 may be an operation to change the tilt of the index Li. For example, when a central region of the index Li is dragged with a pointer device such as a mouse, the position of the index Li may be changed, and when an end region of the index Li is dragged with a pointer device such as a mouse, the tilt of the index Li may be changed. A rotation operation of the three-dimensional image may be performed by adding a pointing operation to a location outside the index Li of the cross-sectional image Da, the cross-sectional image Db, or the cross-sectional image Dc. Depending on the direction and amount of the dragging operation, the tilt of the index Li is changed, and a movement operation to tilt the cross-sectional position is accepted.

[0123] If it is determined in step S41 that no movement operation has been performed, the display process for the detailed observation image D (step S33) is terminated, and if it is determined that a movement operation has been performed, the process proceeds to step S42.

[0124] In step S42, coordinates and orientation corresponding to the movement operation are calculated. The coordinates corresponding to the movement operation are coordinates obtained by shifting the coordinates of the three-dimensional position P1 before the movement by the amount of movement corresponding to the movement operation. The coordinates after the movement operation are converted using the same process as the conversion from the three-dimensional position P1 to the biological feature area corresponding position P2, thereby calculating the coordinates of the biological feature area after the movement operation in the biological feature area-arranged expanded image Eq. Furthermore, the orientation corresponding to the movement operation is an orientation obtained by tilting the orientation perpendicular to any of the cross-sectional images Da, Db, Dc at the three-dimensional position P1 before the movement by the amount of tilt corresponding to the movement operation.

[0125] In the next step S43, the processing unit 30 changes the position of the annotation image Q corresponding to the initial position in the dentition development image Ep to the position after the movement operation. Furthermore, the processing unit 30 generates and displays cross-sectional images Da, Db, and Dc based on the coordinates and orientation after the movement operation. After the movement operation, the cross-sectional images Da, Db, and Dc are generated so that they are centered on the coordinates after the movement operation. Therefore, in the cross-sectional images Da, Db, and Dc, the image before the operation is displayed shifted to the opposite side of the movement operation direction, or an image toward the depth or front of the image before the operation is displayed (see the outline shown by the two-dot chain line in FIG. 15).

[0126] Thereafter, the display process for the detailed observation image D (step S43) is completed.

[0127] If a movement operation is performed again, the processes from step S41 onwards may be repeated.

[0128] In this way, the detailed observation image D is displayed according to the characteristic region determined as the initial position, and the cross-sectional positions of the cross-sectional images Da, Db, and Dc can be adjusted.

[0129] 12 and 14 are performed not only on the characteristic region determined as the initial position, but also on the characteristic region designated by the user and the characteristic region determined according to a predetermined focus order. The process of changing the cross-sectional position as shown in FIG. 14 may be omitted.

[0130] Returning to the explanation based on the flowchart shown in FIG.

[0131] When the process of step S13 is completed, the process proceeds to step S14.

[0132] In step S14, the processing unit 30 determines whether a characteristic region has been designated. The characteristic region may be designated, for example, by a user using a pointer device such as a mouse to select one of a plurality of annotation images Q. The characteristic region may also be designated by specifying an annotation image Q using a switch device such as a keyboard. If it is determined that a characteristic region has been designated, the processing proceeds to step S15, and if it is determined that a characteristic region has not been designated, the processing proceeds to step S18.

[0133] In step S15, annotation change processing is performed. The annotation change processing is processing for changing the annotation image Q in the biological feature area arrangement expanded image Eq so that the position of the feature area displayed as the detailed observation image D can be recognized in the dentition expanded image Ep. In step S15, the detailed observation image D according to the specified feature area is displayed. Therefore, for example, the annotation image Q according to the specified feature area is displayed so as to be distinguished from other annotation images Q. An example of this display may be the same as that in step S20, which will be described later.

[0134] In the next step S16, the designated characteristic region is displayed for observation. This processing is performed, for example, by executing the processing shown in Fig. 12 and Fig. 14 for the designated characteristic region, similar to step S13. As a result, an image D for detailed observation is displayed for the designated characteristic region. After the processing is completed, the process proceeds to step S17.

[0135] In step S17, it is determined whether the diagnostic process has ended. If a process end instruction is input via the user interface 24, 26, etc., the process ends. If the process has not ended, the process returns to step S14 and the subsequent processes are repeated.

[0136] If it is determined in step S14 that no feature region has been designated, the processing from step S18 onwards is executed to execute the following processing in order to indicate that the biological feature region is focused in the focus order in the display of the biological feature region arrangement expanded image Eq by changing the annotation image Q, and to display the detailed observation image D according to the focus of the biological feature region on the display device 22.

[0137] In step S18, it is determined whether or not to automatically switch the focus target. Whether or not to automatically switch the focus target is determined, for example, by an operation via the user interfaces 24, 26. For example, the automatic switching mode may be set in the initial state, and when any key is operated, the mode may be switched to a manual switching mode, which is not automatic switching. If it is determined that the mode is automatic switching, the process proceeds to step S19, and if it is determined that the mode is not automatic switching, the process proceeds to step S24.

[0138] When the process proceeds to step S19, it is determined whether or not the display time has elapsed a predetermined time. The display time may be the display time of the annotation image Q according to the initial position, or the display time of the image for detailed observation D. The predetermined time is an arbitrarily determined time, but may be, for example, a time suitable for detailed observation using the image for detailed observation D. The process of step S19 is repeated until the display time has elapsed the predetermined time, and when it is determined that the display time has elapsed the predetermined time, the process proceeds to step S20.

[0139] In step S20, the processing unit 30 identifies the next feature area after the currently displayed feature area according to the focus order, and changes the annotation image Q so that the next feature area can be distinguished from other feature areas, as shown in FIG. 16.

[0140] For example, as shown in FIG. 16, multiple annotation images Q are superimposed in the biometric feature area arrangement expansion image Eq, and the display of a portion of the multiple annotation images Q that is the focus target may be displayed differently from the other annotation images Q.

[0141] To achieve a different display, for example, the annotation image Q corresponding to the next characteristic region may be displayed brighter than the other annotation images Q.

[0142] The annotation image Q corresponding to the next feature region may be displayed at a blinking speed different from that of the other annotation images Q. Here, the blinking speed is the number of blinks per unit time. A blinking speed of 0 times / hour means that the image does not blink. Different blinking speeds include cases where the image does not blink (i.e., remains lit) and cases where the image blinks. For example, the annotation image Q corresponding to the next feature region may blink, while the other annotation images Q are displayed lit. Alternatively, the annotation image Q corresponding to the next feature region may blink, and the other annotation images Q may also blink at a blinking speed different from that of the annotation image Q corresponding to the next feature region.

[0143] The annotation image Q corresponding to the next characteristic region may be displayed in a color different from the other annotation images Q. For example, the annotation image Q corresponding to the next characteristic region may be displayed in red, and the other annotation images Q may be displayed in orange.

[0144] The annotation image Q corresponding to the next characteristic region may be displayed in a size different from the other annotation images Q. For example, the annotation image Q corresponding to the next characteristic region may be displayed larger than the other annotation images Q (see FIG. 17).

[0145] As described above, the annotation images Q may be distinguished by combining two or more of the differences in brightness, blinking speed, color, and size of the annotation images Q.

[0146] In the developed image Eq of biometric feature region layout, it is not necessary to display a plurality of annotation images Q in a superimposed manner. In the developed image Eq of biometric feature region layout, only the annotation image Q corresponding to the next feature region may be displayed.

[0147] After step S20 is completed, in step S21, observation display processing is performed on the next feature region. The processing of S21 can be performed in conjunction with step S13 above, with the next feature region as the target. As a result, the detailed observation image D of the next feature region is displayed on the display device 22.

[0148] By visually viewing the biological feature area arrangement expanded image Eq and the detailed observation image D displayed on the display device 22, the user can recognize the position of the annotation image Q in the biological feature area arrangement expanded image Eq and observe the detailed observation image D in detail.

[0149] In the next step S22, the processing unit 30 determines whether or not there is a focus order reversal command. If a reversal command is input via the user interface 24, 26, etc., the process proceeds to the next step S23. If a reversal command is not input and it is determined that there is no reversal command, the process proceeds to step S17.

[0150] In step S23, a process for reversing the focus order is executed. For example, a process for reversing the order of the array that determines the focus order is executed. The reversed focus order is stored as the focus order for subsequent processes. After this, the process proceeds to step S17. The determination of whether or not a reversal command is issued in step S22 may be placed further upstream in the process, for example, after step S18, so that if a reversal command is issued, the processes from step S19 onward are carried out according to the instructed focus order.

[0151] By the processing of steps S19 to S21, the biological feature area arrangement expanded image Eq displayed on the display device 22 is changed in accordance with the focus order as a preset time elapses, and the detailed observation image D displayed on the display device 22 is also changed.

[0152] If no feature region is specified and the automatic switching mode is not changed, the processes from steps S19 to S23 are repeated, thereby enabling detailed observation of the biometric feature regions in order without omission or overlap.

[0153] If it is determined in step S18 that the mode is not an automatic switching mode, the process proceeds to step S24.

[0154] In step S24, it is determined whether a next display command has been input. The next display command is a focus target switching command input via the user interfaces 24, 26. For example, a forward display command may be input using the right or down arrow key on the keyboard, and a backward display command may be input using the left or up arrow key. Regardless of whether the next display command is for the backward or forward direction, the process of step S24 is repeated until it is determined that the next display command has been input. If it is determined that the next display command has been input, the process proceeds to the next step S25.

[0155] In the next step S25, the processing unit 30 determines whether the display command is a forward command. If it is determined to be a forward command, the process proceeds to step S26. If a reverse display command has been input, the process proceeds to step S28.

[0156] In step S26, the processing unit 30 identifies the next feature region after the currently displayed feature region in the focus order, and changes the annotation image Q so that the next feature region can be distinguished from the other feature regions (see FIG. 16). The processing in step S26 can be the same as that in step S20.

[0157] After step S26 is completed, in step S27, observation display processing is performed on the next feature region. The processing of S27 can be the same as that of step S13 above, with the next feature region as the target. As a result, the detailed observation image D of the next feature region is displayed on the display device 22.

[0158] By visually viewing the biological feature area arrangement expanded image Eq and the detailed observation image D displayed on the display device 22, the user can recognize the position of the annotation image Q in the biological feature area arrangement expanded image Eq and observe the detailed observation image D in detail.

[0159] If it is determined in step S25 that the command is not a forward direction command, the process proceeds to step S28.

[0160] In step S28, the processing unit 30 identifies the feature region in the reverse order of the currently displayed feature region, i.e., the previous feature region, according to the focus order, and changes the annotation image Q so that the feature region in the reverse order can be distinguished from other feature regions. The processing in step S28 can be the same as that in step S20.

[0161] After step S28 is completed, in step S29, observation and display processing is performed on the feature regions in the reverse order. The processing in step S29 can be the same as that in step S13 above, but with the feature regions in the reverse order as the target. As a result, the detailed observation image D of the feature regions in the reverse order is displayed on the display device 22.

[0162] By visually viewing the biological feature area arrangement expanded image Eq and the detailed observation image D displayed on the display device 22, the user can recognize the position of the annotation image Q in the biological feature area arrangement expanded image Eq and observe the detailed observation image D in detail.

[0163] If no characteristic region is specified and no change is made to the mode that does not perform automatic switching, the processes from step S24 onwards are repeated, so that the annotation image Q changes according to the timing of the user's display command, and the detailed observation image D also changes. At this time, if the user gives a forward command, the biological characteristic regions are displayed in order of focus without omission or overlap.

[0164] At this time, if the user wishes to go back and reconfirm a biometric feature area that has already been observed, the user can give a command in the reverse direction, and the detailed observation image D will be displayed in the reverse direction.

[0165] If the focus order data is defined such that the last biometric feature region returns to the first biometric feature region, after the last biometric feature region is displayed, the first biometric feature region may be displayed according to the focus order.

[0166] In the above example, a case was described in which the process of changing the biological feature area arrangement expanded image and changing the image for detailed observation according to the focus order in response to the passage of a preset time, and the process of performing the process in response to a focus target switching command input through the user interfaces 24 and 26 are used in combination as separate processes.

[0167] However, the processing performed in response to the passage of a preset time and the processing performed in response to a focus target switching command may be combined. For example, on the premise of processing in which the biological feature area arrangement developed image is changed in response to the passage of a preset time and the image for detailed observation is changed, if a focus target switching command is input even before the time has passed, processing in which the biological feature area arrangement developed image is changed in response to the command and the image for detailed observation is changed may be performed.

[0168] In addition, the process of changing the biological feature area arrangement expanded image in accordance with the focus order as a preset time elapses and changing the image for detailed observation may be omitted, or the process of changing the biological feature area arrangement expanded image in accordance with the focus order in response to a focus target switching command may be omitted.

[0169] <Effects, etc.> According to the image processing device 20, image processing method, and program 34a configured as described above, the fact that biometric feature regions are focused in focus order is indicated by a change in the annotation image Q, and detailed observation images D corresponding to the focus of the biometric feature regions are displayed on the display device 22. Therefore, by viewing the biometric feature region arrangement expanded image Eq, it is easy to sequentially recognize multiple biometric feature regions. Furthermore, since detailed observation images D corresponding to the focus order are displayed on the display device 22, the focused biometric feature regions can be observed in detail. Compared to when biometric feature regions are individually specified, it is easy to recognize the biometric feature regions in the focus order without omission or overlap. Even when multiple biometric feature regions are located closely together, it is easy to recognize and observe the adjacent multiple biometric feature regions as separate biometric feature regions.

[0170] Furthermore, the dentition expansion image Ep is an image of the entire area of ​​the tissues E that constitute the dentition, viewed from the front, and the annotation image Q is displayed on this image. In this case, it is easy to recognize the position of the biological characteristic area within the entire area of ​​the tissues E that constitute the dentition.

[0171] Furthermore, a biological feature area arrangement expanded image Eq in which a plurality of annotation images Q are superimposed on the row of teeth expanded image Ep is displayed on the display device 22. Then, a portion of the plurality of annotation images Q that is the focus target is displayed differently from the other annotation images Q. This allows the position of the focused biological feature area in the row of teeth expanded image Ep to be easily recognized, while the focused biological feature area can be observed in detail.

[0172] In this case, if the annotation image Q that is the focus target is displayed in a blinking manner, the focused biometric feature area can be easily recognized.

[0173] Furthermore, if the focused annotation image Q is displayed at a different blinking speed, color, or size from the other annotation images Q, the focused annotation image can be made to stand out.

[0174] If the focus order is determined based on the multiple biological feature region corresponding positions P2 in the dentition expansion image Ep, the focus order can be determined, for example, based on the widthwise or vertical positions of the biological feature region corresponding positions. If the focus order is determined based on positional regularity, the focus order is easy to predict. Therefore, the detailed observation images D can be observed sequentially while predicting the focus order, making the observation work easier.

[0175] By changing the biological feature area arrangement expanded image Eq displayed on the display device 22 in accordance with the focus order as a preset time elapses, and by changing the detailed observation image D displayed on the display device 22, the focus target can be automatically changed as time elapses.

[0176] It would also be convenient if the focus order could be reversed in response to a reversal command input through the user interfaces 24 and 26. For example, this would be easy to handle when observation is made too quickly or when the user wishes to go back and observe.

[0177] Furthermore, in response to a focus target switching command, the biological feature area arrangement expanded image Eq displayed on the display device 22 is changed in accordance with the focus order, and the image for detailed observation D displayed on the display device 22 is also changed, so the focus target can be changed based on input to the user interface. Therefore, the image for detailed observation D can be changed according to the progress of the user's observation. Furthermore, even if there are adjacent biological feature areas, they can be easily distinguished.

[0178] Furthermore, if the detailed observation image D includes three cross-sectional images Da, Db, and Dc that are orthogonal to one another, it is easy to observe the biological characteristic region in detail.

[0179] Furthermore, the cross-sectional position of at least one of the three cross-sectional images Da, Db, and Dc can be changed in response to a movement operation of the index Li via the user interfaces 24 and 26. Therefore, by changing the cross-sectional positions of the cross-sectional images Da, Db, and Dc, it is easy to observe the biological characteristic region.

[0180] Simply performing a movement operation on the index Li will not change the position of the annotation image Q in the dentition unfolded image Ep, but after the movement operation, an operation to change the position of the annotation image Q can be performed to accept the change and change the position of the annotation image Q in the dentition unfolded image Ep.

[0181] The operation of changing the position of the annotation image Q may be performed, for example, by an operation on the detailed observation image D. It may also be performed by an operation using the index Li. The change of the position of the annotation image Q may also be called an annotation position change. The operation of changing the annotation position may also be called an annotation position change operation. The annotation position change operation is an operation that converts an operation on the X1Y1Z1 coordinates into a change of the annotation position on the RT coordinates or a change of the annotation position on the RST coordinates. In addition, the annotation position change operation is an operation that converts an operation on the captured image data 3D image into a change of the annotation position on the tooth arch expansion area coordinates.

[0182] Furthermore, the position of the annotation image Q in the dentition development image Ep is changed in response to a movement operation on the index Li via the user interfaces 24, 26. This makes it easy to grasp the position of the detailed observation image in the dentition development image Ep after the movement operation.

[0183] Furthermore, because the detailed observation image D includes a cross-sectional image Da that views the biometric characteristic area from the front and two cross-sectional images that are perpendicular to the first image, the biometric characteristic area is easy to observe. In particular, by using the cross-sectional image Da that views the biometric characteristic area from the front, it is easy to grasp the position and orientation of the biometric characteristic area relative to the teeth.

[0184] <Modification> Various modifications based on the above embodiment will be described.

[0185] As in a first modified example shown in FIG. 18, the processing unit 30 may determine the focus order based on the importance of the biometric feature regions.

[0186] For example, as described above, if the biological feature region is a lesion region, the importance of the lesion region may be determined based on at least one of the progression level, the lesion location, the risk according to the disease name, and the confidence level.

[0187] For example, since the degree of progress is thought to be positively correlated with the size of the detected biometric feature area, it is conceivable that the larger the biometric feature area, the higher the importance level will be, using the size of the detected biometric feature area as a parameter.

[0188] It is conceivable that the importance of a lesion site increases the closer it is to the tooth root from the tooth surface. Therefore, the positional relationship between the detected biometric feature area and the tooth or jawbone can be used as a parameter to identify the lesion site and determine its importance.

[0189] Furthermore, regarding the disease name, it may be considered to give a higher importance to cancer than to tooth decay and periodontal disease, for example. The disease name of a biometric feature region may be estimated by applying, for example, a trained machine learning model. For example, a large amount of data in which image data of a biometric feature region is labeled with a disease name is prepared as training data. A machine learning model trained to estimate the disease name of a biometric feature region is prepared using the training data. The disease name can be estimated by applying the trained machine learning model to the biometric feature region.

[0190] In this case, the focus order is determined according to the importance, and therefore the focus order may be determined randomly in the row of teeth expanded image Ep as shown in Fig. 18. For example, when characteristic areas are observed in the row of teeth expanded image Ep in the focus order, they may be focused in an order from bottom to top or from right to left.

[0191] According to this modification, the focus order can be set according to the importance, and therefore the detailed observation images D can be observed in order of their perceived importance.

[0192] In the above embodiment, an example has been described in which the biometric feature region is the lesion region L. As in the biometric feature region-arranged expanded image Eqa of the second modified example shown in FIG. 19 , the biometric feature region need only be an image-characteristic region in the dentition-forming tissue E, and does not have to be the lesion region L. For example, the biometric feature region may be a plurality of common biometric feature regions having a common biometric feature. Specifically, the biometric feature region may be the root apex Lb of a tooth. In this case, an annotation image Q is displayed at each root apex Lb of a plurality of teeth T in the dentition-forming tissue E. A focus order is determined for the plurality of root apexes Lb. The focus order is determined, for example, based on their positions in the dentition-forming tissue E. For example, the focus order is determined so that the upper dentition is placed higher than the lower dentition, and the right dentition is placed higher than the left dentition.

[0193] According to the determined focus order, as described in the above embodiment, the annotation image Q changes so that the focused root apex Lb is identified, allowing the user to observe the multiple root apexes Lb in detail in sequence in the focus order while grasping the position of each apex Lb.

[0194] According to this modification, in a plurality of common biometric feature regions, some of the plurality of annotation images Q are focused in focus order, which makes it easier to observe the common biometric features.

[0195] The common biometric feature area may be the mandibular canal.

[0196] 20 , for a unique root apex Lb among annotation images Q corresponding to a plurality of root apexes Lb, an annotation image Qq different from the other annotation images Q may be displayed regardless of whether it is focused. The unique root apex Lb is, for example, a location where a lesion such as periodontal disease has been detected. In this case, the annotation image Qq and the other annotation images Q may be displayed in different colors, shapes, blinking speeds, or sizes, for example.

[0197] For example, the annotation image Qq corresponding to the peculiar root apex Lb and the annotation images Q corresponding to the other root apex Lb may be displayed in different colors, and the focused root apex Lb may be displayed in a blinking manner. This allows the user to sequentially observe the multiple root apex Lb in detail while recognizing the peculiar root apex Lb.

[0198] Furthermore, multiple annotation images Q that are not the focus target may be displayed in the same manner regardless of whether they have a unique feature. When they are focused, they may be displayed differently depending on whether they have a unique feature. For example, annotation images Q that are not the focus target may be displayed in a blinking manner, and focused annotation images Q may be displayed in a blinking manner. In this case, annotation images Q corresponding to root apexes Lb that have no unique feature may be displayed in the same color as annotation images Q that are not the focus target, and annotation images Q corresponding to root apexes Lb that have a unique feature may be displayed in a different color from annotation images Q that are not the focus target. This allows multiple root apexes Lb to be sequentially observed in detail while recognizing the unique root apex Lb.

[0199] In the above embodiment, an example has been described in which each position of the structures E constituting the dentition is identified based on three-dimensional volume data, and the identified structures E constituting the dentition are developed into an expanded image Ep of the dentition.

[0200] However, as a fourth modified example, instead of individually identifying the positions of the structures E that constitute the dentition based on the three-dimensional volume data, the dentition may be developed into an expanded image Ep of the dentition based on a standard region of the structures E that constitute the dentition.

[0201] For example, as shown in Fig. 21, when performing CT imaging with the CT imaging device 10, the head H is positioned at a fixed position by the head holder 9. As a result, the dentition-forming tissues E are also positioned at a fixed position relative to the head holder 9, as shown in Fig. 22.

[0202] The positional relationship between the head holder 9 and the X-ray generator 11 and X-ray detector 12 can be a known positional relationship. Furthermore, the area in the head H where the tissues constituting the dentition E extend can be considered to be more or less constant. For this reason, in the three-dimensional volume data based on CT imaging data, the area of ​​the standard tissues constituting the dentition E can be treated as the area of ​​the actual tissues constituting the dentition E.

[0203] Furthermore, assuming a standard tissue forming the dentition E, it is possible to determine in advance a general-purpose conversion process for converting the standard tissue forming the dentition E into a dentition expansion image Ep. The region of the three-dimensional volume data where the tissue forming the dentition E is expected to exist can be converted into a dentition expansion image Ep by the general-purpose conversion process. Furthermore, by performing the general-purpose conversion process on the biometric feature region detected in the three-dimensional volume data, it is possible to calculate the corresponding position of the biometric feature region in the dentition expansion image Ep.

[0204] In the above embodiment, the case where the image processing device 20 is used in combination with the CT imaging device 10 has been described.

[0205] The image processing device 20 can also be configured as a device separate from the CT imaging device 10.

[0206] In this case, like the image processing device 120 according to the fifth modification shown in FIGS. 23 and 24, the image processing device 120 may include a data access device 130 that is connected by turning the connection port 36. The data access device 130 is, for example, a device that can read data recorded on a recording medium 128. The recording medium 128 may be an optical recording medium such as an optical disc, or may be a flash memory such as a USB memory or an SD card. The data access device 130 may be an optical disc reader or a card reader.

[0207] Image data obtained by CT imaging is recorded on the recording medium 128. The image data may be a group of X-ray projection image data, three-dimensional volume data, or a data group of multiple tomographic images.

[0208] The image processing device 120 can execute the same processing as in the above embodiment by reading image data obtained by CT imaging via the data access device 130.

[0209] Image data obtained by CT scanning may be stored in a server device, and the image processing device 120 may access the server device via a communication network to obtain the image data obtained by CT scanning. In this case, the server device may be located within the facility where the image processing device 120 is located, or may be located outside the facility. The server device may also be a cloud server. The image processing device 120 can access the server device via a dedicated line or a public line network to obtain the image data obtained by CT scanning.

[0210] {Other variations} In the above embodiment and various modified examples, other information may be displayed around the biological feature area arrangement expanded image Eq or the detailed observation image D. For example, the name of a disease may be written. For example, when multiple lesion areas L are detected, the disease name of each of the multiple lesion areas may be displayed. The disease name may be displayed together with the cross-sectional image.

[0211] The configurations described in the above embodiment and modifications can be combined as appropriate as long as they are not mutually contradictory.

[0212] The present disclosure discloses the following aspects.

[0213] The first aspect is an image processing device that processes image data obtained by CT imaging of tissues forming the dentition to generate an image for diagnosis, and is equipped with a storage device that stores the image data, a processing device, and a display device that displays an image for observation based on the output of the processing device, wherein the processing device generates three-dimensional image data of the tissues forming the dentition based on the image data, detects a plurality of biological characteristic areas in the tissues forming the dentition based on the image data, identifies a plurality of three-dimensional positions where each of the plurality of biological characteristic areas exists in a coordinate system of the three-dimensional image data, generates an expanded image of the dentition in which the tissues forming the dentition are expanded based on the three-dimensional image data, identifies a plurality of biological characteristic area corresponding positions in the expanded image of the dentition that correspond to each of the plurality of three-dimensional positions, and The image processing device displays, on the display device, a biometric feature area layout expanded image in which at least one of the plurality of annotation images is superimposed on the dentition expanded image so that an annotation image showing each of the plurality of biometric feature areas is located at a corresponding position among the plurality of biometric feature area corresponding positions, generates detailed observation images at each of the plurality of three-dimensional positions based on the three-dimensional image data, determines a focus order of the plurality of biometric feature areas according to a predetermined rule, indicates that the biometric feature areas are focused in the focus order by a change in the annotation image in the display device when displaying the biometric feature area layout expanded image, and displays the detailed observation images according to the focus of the biometric feature areas on the display device.

[0214] According to this display device, the annotation image changes to indicate that the biometric feature regions are focused in the focus order, and detailed observation images corresponding to the focus of the biometric feature regions are displayed on the display device. Therefore, by viewing the biometric feature region layout expanded image, it is easy to sequentially recognize the multiple biometric feature regions. Furthermore, because detailed observation images corresponding to the focus image are displayed on the display device, the focused biometric feature region can be observed in detail. Therefore, it is easy to recognize the multiple biometric feature regions without any omissions.

[0215] A second aspect is an image processing device according to the first aspect, in which the processing device displays, on the display device, the biological feature area arrangement expanded image in which the plurality of annotation images are superimposed on the dentition expanded image, and displays a portion of the plurality of annotation images that are the subject of the focus in a different manner from the other annotation images.

[0216] In this case, the annotation image that is the focus target is displayed differently from the other annotation images, so that the position of the focused biometric characteristic area can be easily recognized and the focused biometric characteristic area can be observed in detail.

[0217] A third aspect is the image processing device according to the first or second aspect, wherein the processing device blinks the display of a part of the plurality of annotation images that are the focus target.

[0218] As a result, the annotation image that is the focus target is displayed in a blinking manner, making it easier to recognize the focused biometric feature area.

[0219] A fourth aspect is an image processing device according to the first or second aspect, wherein the processing device displays a portion of the plurality of annotation images that is the focus target at a different blinking speed, color, or size than the other annotation images.

[0220] This allows the focused annotation image to stand out by displaying it at a different blinking speed, in a different color, or in a different size from the other annotation images.

[0221] A fifth aspect is an image processing device according to any one of the first to fourth aspects, further comprising a user interface, wherein the processing device changes the biological feature area arrangement expanded image displayed on the display device in accordance with the focus order in response to a focus target switching command input through the user interface, and changes the detailed observation image displayed on the display device.

[0222] This allows the focus target to be changed based on input to the user interface.

[0223] A sixth aspect is an image processing device according to any one of the first to fifth aspects, wherein the processing device changes the biological feature area arrangement expanded image displayed on the display device in accordance with the focus order as a preset time elapses, and changes the image for detailed observation displayed on the display device.

[0224] This allows the focus target to be changed over time.

[0225] A seventh aspect is an image processing device according to any one of the first to sixth aspects, further comprising a user interface, wherein the processing device reverses the focus order in response to a reverse command input through the user interface.

[0226] This is convenient because it allows you to reverse the focus order.

[0227] An eighth aspect is an image processing device according to any one of the first to seventh aspects, wherein the processing device determines the focus order based on the positions corresponding to the plurality of biological feature regions in the dental arch unfolded image.

[0228] This allows the focus order to be determined based on the positions corresponding to a plurality of biometric feature regions in the dentition expansion image, for example, based on the widthwise or vertical positions of the biometric feature region corresponding positions.

[0229] A ninth aspect is an image processing device according to any one of the first to eighth aspects, wherein the processing device detects a plurality of common biometric feature areas having common biometric features as the plurality of biometric feature areas.

[0230] As a result, in a plurality of common biometric feature regions, some of the plurality of annotation images are focused in the focus order, which makes it easier to observe the common biometric feature.

[0231] A tenth aspect is an image processing device according to any one of the first to ninth aspects, wherein the processing device sets an importance level for each of the plurality of biometric feature areas based on the respective features, and determines the focus order based on the importance level for each of the plurality of biometric feature areas.

[0232] This allows you to set the focus order according to importance.

[0233] An eleventh aspect is an image processing device according to any one of the first to tenth aspects, wherein the processing device generates an image of the entire area of ​​the tissues that make up the dentition as the dentition expansion image.

[0234] This allows an annotation image showing the biological characteristic area to be displayed on an image of the entire area of ​​the tissues that make up the dentition, making it easy to recognize the position of the biological characteristic area within the entire area of ​​the tissues that make up the dentition.

[0235] The processing device may set a dentition-forming tissue region, which is the region in which the dentition-forming tissues extend, and a dentition-formed region in which the dentition-forming tissue region is expanded, and the expansion of the dentition-forming tissues may be performed by an expansion process in which image data of the dentition-forming tissues in the dentition-forming tissue region is expanded to fit the shape of the dentition-formed region, thereby enabling appropriate image data processing for visual recognition of biological feature regions.

[0236] The unfolding process may be performed so that the buccal-lingual direction in the curved state before unfolding, which is the state of the tissue region constituting the dentition before unfolding, corresponds to the normal viewing direction in the view state after unfolding, which is the state of the unfolded dentition region after unfolding. This makes it easy to understand the position of the biological feature region.

[0237] The dentition-forming tissue region may be set to have a thickness in the buccolingual direction, and the dentition-unfolded region may be set to have a thickness in the normal viewing direction corresponding to the buccolingual direction, thereby enabling recognition of a biometric feature region within an appropriate range.

[0238] The expansion may involve performing an original coordinate calculation to associate the biometric feature region with the three-dimensional position of the biometric feature region corresponding position. The original coordinate calculation may involve specifying the position of the three-dimensional position of the biometric feature region in the buccolingual direction, and performing a calculation to specify the position of the biometric feature region corresponding position in the normal viewing direction corresponding to the buccolingual direction. This makes it possible to accurately calculate the three-dimensional position of the biometric feature region and the biometric feature region corresponding position.

[0239] A twelfth aspect is the image processing device according to any one of the first to eleventh aspects, wherein the detailed observation image includes three cross-sectional images that are orthogonal to each other.

[0240] This makes it easier to observe the biometric feature area in detail.

[0241] A thirteenth aspect is an image processing device according to the twelfth aspect, further comprising a user interface, wherein each of the three cross-sectional images includes an indicator indicating the position of the other cross-section, and the processing device changes the cross-sectional position of at least one of the three cross-sectional images in response to a movement operation on the indicator via the user interface.

[0242] This makes it easier to observe the biological characteristic area by changing the cross-sectional position of the detailed observation image.

[0243] The processing device may perform processing to distinguish between the dentition region and the alveolar bone region based on the three-dimensional image data, and perform image processing to display the dentition region and the alveolar bone region in different modes in the dentition expansion image. This allows the dentition region and the alveolar bone region to be displayed separately in the dentition expansion image, making it easier to grasp the position of the biometric feature region.

[0244] The processing device may use different colors for the dentition region and the alveolar bone region in the dentition expansion image, thereby making it easier to distinguish and recognize the dentition region and the alveolar bone region.

[0245] The processing device may set a transparency for the alveolar bone region that allows the dentition region to be seen through, thereby enabling the observation of a biological characteristic region for the portion of the dentition that is inside the alveolar bone while grasping the positional relationship between the alveolar bone and the dentition.

[0246] The processing device may be configured to display the functional tissues that make up the dentition of a living body by a boundary surface between presence and absence, and to perform gaze direction-responsive transparency image processing, which is image processing in which the transparency of the boundary surface changes depending on the gaze direction, thereby making it easier to visually recognize the shapes of the functional tissues that make up the dentition.

[0247] When the element that increases transparency is a parent transparency element, and increasing transparency is parent transparency, and the element that decreases transparency is an anti-transparency element, and decreasing transparency is anti-transparency, the gaze direction-compatible transparency image processing may be a combination of at least one of the following group A and at least one of the following group B. Group A: Processing to enlarge or increase the parent transparency element of the boundary surface of normal vision Processing to promote parent transparency of the boundary surface of normal vision Treatment to reduce, diminish or eliminate the anti-transparency elements of the interface of normal vision Treatment to suppress the anti-transparency of the boundary surface of normal vision Group B: Processing to reduce, decrease or eliminate the parent transparency elements of the boundary surface in oblique or side views - Processing to suppress parent transparency of the boundary surface in oblique or side view Treatment to enlarge or increase the anti-transparency factor of the said boundary surface in oblique or lateral view Treatment to promote anti-transparency of the boundary surface in oblique or lateral views This makes it easier to visually recognize the three-dimensional shapes of the functional tissues that constitute the dentition.

[0248] A fourteenth aspect is an image processing method for generating an image for diagnosis by processing image data obtained by CT imaging of tissues constituting the dentition, which generates three-dimensional image data of the tissues constituting the dentition based on the image data, detects a plurality of biological characteristic areas in the tissues constituting the dentition based on the image data, identifies a plurality of three-dimensional positions in the coordinate system of the three-dimensional image data where each of the plurality of biological characteristic areas exists, generates an expanded image of the dentition in which the tissues constituting the dentition are expanded based on the three-dimensional image data, identifies a plurality of positions in the expanded image of the dentition corresponding to each of the plurality of three-dimensional positions, and generates annotation images showing each of the plurality of biological characteristic areas. a biometric feature area arrangement expanded image in which at least one of the plurality of annotation images is superimposed on the dentition expanded image so that the corresponding one of the plurality of biometric feature area corresponding positions is located at a corresponding position among the plurality of biometric feature area corresponding positions, a detailed observation image at each of the plurality of three-dimensional positions is generated based on the three-dimensional image data, a focus order of the plurality of biometric feature areas is determined according to a predetermined rule, in displaying the biometric feature area arrangement expanded image, the annotation image changes to indicate that the biometric feature areas are focused in the focus order, and the detailed observation image according to the focus of the biometric feature areas is displayed on the display device.

[0249] According to this image processing method, the annotation image changes to indicate that the biometric feature regions are focused in focus order, and a detailed observation image corresponding to the focus of the biometric feature region is displayed on the display device. By viewing the biometric feature region layout expanded image, it is easy to sequentially recognize multiple biometric feature regions. Furthermore, because a detailed observation image corresponding to the focus image is displayed on the display device, the focused biometric feature region can be observed in detail. Therefore, it is easy to recognize multiple biometric feature regions without omission.

[0250] The fifteenth aspect is a program for generating an image for diagnosis by processing image data obtained by CT scanning of tissues constituting the dentition, the program causing a computer to generate three-dimensional image data of the tissues constituting the dentition based on the image data, detect a plurality of biological characteristic areas in the tissues constituting the dentition based on the image data, identify a plurality of three-dimensional positions in the coordinate system of the three-dimensional image data where each of the plurality of biological characteristic areas exists, generate an expanded image of the dentition in which the tissues constituting the dentition are expanded based on the three-dimensional image data, identify a plurality of positions in the expanded image of the dentition corresponding to the plurality of three-dimensional positions, and generate an annotation image showing each of the plurality of biological characteristic areas based on the plurality of three-dimensional positions. the program executes a process of: displaying, on a display device, a biometric feature area arrangement expanded image in which at least one of the plurality of annotation images is superimposed on the dentition expanded image so that at least one of the plurality of annotation images is positioned at a corresponding position out of a number of biometric feature area corresponding positions; generating detailed observation images at each of the plurality of three-dimensional positions based on the three-dimensional image data; determining a focus order of the plurality of biometric feature areas in accordance with a predetermined rule; indicating, in the display of the biometric feature area arrangement expanded image, that the biometric feature areas are focused in the focus order by a change in the annotation image; and displaying, on the display device, the detailed observation images according to the focus of the biometric feature areas.

[0251] According to this program, the annotation image changes to indicate that the biometric feature regions are being focused in focus order, and a detailed observation image corresponding to the focus of the biometric feature region is displayed on the display device. Therefore, by viewing the biometric feature region layout expanded image, it is easy to sequentially recognize multiple biometric feature regions. Furthermore, because a detailed observation image corresponding to the focus image is displayed on the display device, the focused biometric feature region can be observed in detail. Therefore, it is easy to recognize multiple biometric feature regions without omission.

[0252] The above description is illustrative in all respects and is not intended to limit the scope of the present invention. It is understood that numerous variations not illustrated can be envisaged without departing from the scope of the present invention. [Explanation of symbols]

[0253] 10 CT imaging device 20, 120 Image processing device 22 Display device 24, 26 User Interface 30 processing units 32 Arithmetic circuit 34 Storage device 34a Program 34b Data D. Detailed observation image Da, Db, Dc cross-sectional images E. Tissues that make up the dentition Ep dental arch expansion image Eq, Eqa Biometric feature area layout expansion image L Lesion area Lb root apex Li, Lia indicators P1 3D position P2 Biometric feature area corresponding position Q, Qq annotation images T tooth

Claims

1. An image processing device that processes image data obtained by CT imaging of tissues constituting the dentition to generate an image for diagnosis, a storage device that stores the image data; a processing device; a display device that displays an image for observation based on the output of the processing device; Equipped with The processing device includes: generating three-dimensional image data of the tissues constituting the dentition based on the image data; detecting a plurality of biometric characteristic regions in the tissues constituting the dentition based on the image data; identifying a plurality of three-dimensional positions in a coordinate system of the three-dimensional image data where the plurality of biometric feature regions exist, generating a dental arch expanded image in which the dental arch constituent tissues are expanded based on the three-dimensional image data; identifying a plurality of biometric feature region corresponding positions in the dentition expansion image, each corresponding to the plurality of three-dimensional positions; displaying, on the display device, a biometric feature area arrangement expanded image in which at least one of the plurality of annotation images is superimposed on the dentition expanded image so that an annotation image showing each of the plurality of biometric feature areas is positioned at a corresponding position among the plurality of biometric feature area corresponding positions; generating detailed observation images at each of the plurality of three-dimensional positions based on the three-dimensional image data; determining a focus order for the plurality of biometric feature regions according to a predetermined rule; an image processing device that, in displaying the biometric feature area layout expanded image, indicates by a change in the annotation image that the biometric feature area is focused in the focus order, and displays the detailed observation image according to the focus of the biometric feature area on the display device.

2. 2. The image processing device according to claim 1, The processing device includes: displaying, on the display device, the biometric feature area arrangement expanded image in which the plurality of annotation images are superimposed on the dentition expanded image; The image processing device displays a part of the plurality of annotation images that is the focus target in a manner different from other annotation images.

3. 3. The image processing device according to claim 1, The processing device includes: The image processing device causes a part of the plurality of annotation images that is the focus target to be displayed in a blinking manner.

4. 3. The image processing device according to claim 1, The processing device includes: an image processing device that displays a part of the plurality of annotation images that is the focus target at a different blinking speed, color, or size from the other annotation images;

5. 3. The image processing device according to claim 1, further comprising a user interface; The processing device includes: an image processing device that changes the biological feature area arrangement expanded image displayed on the display device in accordance with the focus order in response to a focus target switching command input through the user interface, and changes the image for detailed observation displayed on the display device.

6. 3. The image processing device according to claim 1, The processing device includes: an image processing device that changes the biological feature area arrangement expanded image displayed on the display device in accordance with the focus order as a preset time elapses, and changes the image for detailed observation displayed on the display device.

7. 3. The image processing device according to claim 1, further comprising a user interface; The processing device includes: The image processing device changes the focus order to a reverse order in response to a reverse command input through the user interface.

8. 3. The image processing device according to claim 1, The processing device includes: an image processing device that determines the focus order based on the plurality of biological feature region corresponding positions in the dentition expansion image.

9. 3. The image processing device according to claim 1, The processing device includes: an image processing device that detects, as the plurality of biometric feature regions, a plurality of common biometric feature regions having a common biometric feature;

10. 3. The image processing device according to claim 1, The processing device includes: assigning a level of importance to each of the plurality of biometric feature regions based on the respective features; The image processing device determines the focus order based on the importance of each of the plurality of biometric feature regions.

11. 3. The image processing device according to claim 1, The processing device includes: an image processing device that generates, as the dentition expansion image, an image of the entire area of ​​the tissues that constitute the dentition viewed from the front;

12. 3. The image processing device according to claim 1, An image processing device, wherein the detailed observation image includes three cross-sectional images that are orthogonal to each other.

13. 13. The image processing device according to claim 12, further comprising a user interface; each of the three cross-sectional images includes an index indicating the position of each of the other cross-sectional images; The processing device includes: an image processing device that changes a cross-sectional position of at least one of the three cross-sectional images in response to a movement operation on the index through the user interface;

14. An image processing method for generating a diagnostic image by processing image data obtained by CT imaging of tissues constituting a dental arch, comprising: generating three-dimensional image data of the tissues constituting the dentition based on the image data; detecting a plurality of biometric characteristic regions in the tissues constituting the dentition based on the image data; identifying a plurality of three-dimensional positions in a coordinate system of the three-dimensional image data where the plurality of biometric feature regions exist, generating a dental arch expanded image in which the dental arch constituent tissues are expanded based on the three-dimensional image data; identifying a plurality of biometric feature region corresponding positions in the dentition expansion image, each corresponding to the plurality of three-dimensional positions; displaying, on a display device, a biometric feature area arrangement expanded image in which at least one of the plurality of annotation images is superimposed on the dentition expanded image so that an annotation image showing each of the plurality of biometric feature areas is positioned at a corresponding position among the plurality of biometric feature area corresponding positions; generating detailed observation images at each of the plurality of three-dimensional positions based on the three-dimensional image data; determining a focus order for the plurality of biometric feature regions according to a predetermined rule; an image processing method in which, in displaying the biometric feature area layout expanded image, the annotation image is changed to indicate that the biometric feature areas are focused in the focus order, and the detailed observation image corresponding to the focus of the biometric feature areas is displayed on the display device.

15. A program for generating diagnostic images by processing image data obtained by CT imaging of tissues constituting the dental arch, On the computer, generating three-dimensional image data of the tissues constituting the dentition based on the image data; detecting a plurality of biometric characteristic regions in the tissues constituting the dentition based on the image data; identifying a plurality of three-dimensional positions in a coordinate system of the three-dimensional image data where the plurality of biometric feature regions exist, generating a dental arch expanded image in which the dental arch constituent tissues are expanded based on the three-dimensional image data; identifying a plurality of biometric feature region corresponding positions in the dentition expansion image, each corresponding to the plurality of three-dimensional positions; displaying, on a display device, a biometric feature area arrangement expanded image in which at least one of the plurality of annotation images is superimposed on the dentition expanded image so that an annotation image showing each of the plurality of biometric feature areas is positioned at a corresponding position among the plurality of biometric feature area corresponding positions; generating detailed observation images at each of the plurality of three-dimensional positions based on the three-dimensional image data; determining a focus order for the plurality of biometric feature regions according to a predetermined rule; a program for executing a process of indicating, in the display of the biometric feature area layout expansion image, that the biometric feature areas are focused in the focus order by changing the annotation image, and displaying, on the display device, the image for detailed observation according to the focus of the biometric feature areas.

Images