Three-dimensional bone model display apparatus, three-dimensional bone model display method, and program

Abstract

A three-dimensional bone model display apparatus (1) is provided with: a storage device (12) for holding three-dimensional data indicating a three-dimensional bone model; an imaging unit (13) for imaging, among sites of an animal, a site having a bone therein; a position identification unit (141) for identifying the position of the bone at the site on the basis of a site image obtained by imaging the site by using the imaging unit (13); an augmented reality image generation unit (142) for reading the three-dimensional data from the storage device (12) and generating and outputting an augmented reality image in which the three-dimensional bone model indicated by the read three-dimensional data is disposed at a position in a space corresponding to the position identified by the position identification unit (141); and a display unit (15) for displaying the augmented reality image outputted from the augmented reality image generation unit (142).

Description

Three-dimensional bone model display device, three-dimensional bone model display method, and program

[0001] The present disclosure relates to a three-dimensional bone model display device and the like that displays an augmented reality image in which a three-dimensional bone model that resembles an animal's bone is projected onto space as a virtual solid object.

[0002] In recent years, as an example of a surgical support system, development has been progressing on a surgical navigation system that utilizes augmented reality (AR) and mixed reality (MR) technologies. For example, Patent Document 1 discloses a surgical support terminal used in a surgery to perform osteotomy of a femur or tibia and install an implant. Specifically, the surgical support terminal discloses a surgical support terminal that displays a virtually represented augmented reality image superimposed on a captured image of a patient's femur or tibia, which is a lesion site.

[0003] Patent No. 7012302

[0004] However, the surgical support terminal disclosed in Patent Document 1 is merely a surgical support system that utilizes technology for displaying an augmented reality image on a display that is located at a position unrelated to the spatial position of the actual bone. In this surgery, the surgeon is required to recognize the exact position of the bone to be manipulated in the surgical field and perform osteotomy on the bone. Therefore, the surgical support terminal that displays an augmented reality image at a position unrelated to the spatial position of the actual bone may not be able to provide sufficient support to the surgeon during the surgery.

[0005] Therefore, the present disclosure aims to provide a three-dimensional bone model display device, etc., that can provide more thorough support to surgeons during surgery by superimposing a three-dimensional bone model on the bones of the target patient in the actual surgical field.

[0006] A three-dimensional bone model display device according to one aspect of the present disclosure is a three-dimensional bone model display device that displays an augmented reality image in which a three-dimensional bone model modeled on an animal's bone is projected onto space as a virtual solid object, and includes: a storage device that stores three-dimensional data representing the three-dimensional bone model; an imaging unit that captures an image of a part of the animal that has the bone inside; a position identification unit that identifies the position of the bone in the part based on a part image obtained by the imaging unit capturing the part; an augmented reality image generation unit that reads the three-dimensional data from the storage device, generates and outputs an augmented reality image in which the three-dimensional bone model represented by the read three-dimensional data is positioned in a position in space corresponding to the position of the bone identified by the position identification unit; and a display unit that displays the augmented reality image output from the augmented reality image generation unit.

[0007] A three-dimensional bone model display method according to one aspect of the present disclosure is a three-dimensional bone model display method executed by a three-dimensional bone model display device that displays an augmented reality image in which a three-dimensional bone model modeled on an animal's bone is projected onto space as a virtual solid object, wherein the three-dimensional bone model display device includes a storage device that stores three-dimensional data representing the three-dimensional bone model, and an imaging unit that captures an image of a part of the animal that has the bone inside. The three-dimensional bone model display method includes a position identification step that identifies the position of the bone in the part based on a part image obtained by the imaging unit capturing the part; an augmented reality image generation step that reads the three-dimensional data from the storage device, and generates and outputs an augmented reality image in which the three-dimensional bone model represented by the read three-dimensional data is positioned in a position in space corresponding to the position of the bone identified in the position identification step; and a display step that displays the augmented reality image output in the augmented reality image generation step.

[0008] A program according to one aspect of the present disclosure causes a computer to execute the above-described three-dimensional bone model display method.

[0009] According to the present disclosure, a three-dimensional bone model display device and the like are provided that can provide more thorough support to the surgeon during surgery.

[0010] Fig. 1 is a block diagram showing the configuration of a three-dimensional bone model display device according to an embodiment. Fig. 2 is a flowchart showing the operation of the three-dimensional bone model display device according to an embodiment. Fig. 3 is a diagram showing an example of an augmented reality image displayed on a display unit when a surgeon views a bone to be operated on from above (palmar side). Fig. 4 is a diagram showing a scene in which the augmented reality image shown in Fig. 3(b) is confirmed. Fig. 5 is a diagram showing an example of an augmented reality image displayed on a display unit when a surgeon views a bone to be operated on from the side (radial side).

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Note that each of the embodiments described below represents a specific example of the present disclosure. Therefore, the numerical values, shapes, materials, components, component placement positions, connection forms, etc. shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Therefore, among the components in the following embodiments, components that are not recited in the independent claims of the present disclosure will be described as optional components.

[0012] Note that each figure is a schematic diagram and is not necessarily an exact illustration. Therefore, the scales and the like do not necessarily match in each figure. Furthermore, in each figure, the same reference numerals are used for substantially the same configurations, and redundant explanations will be omitted or simplified.

[0013] Furthermore, in this specification, the terms "up" and "down" do not necessarily refer to the upward direction (vertically upward) and downward direction (vertically downward) in absolute spatial recognition.

[0014] (Embodiment) Hereinafter, a three-dimensional bone model display device according to this embodiment will be described.

[0015] [Configuration] First, the configuration of the three-dimensional bone model display device according to the present embodiment will be described. Fig. 1 is a block diagram showing the configuration of a three-dimensional bone model display device 1 according to the embodiment.

[0016] As shown in Fig. 1, the 3D bone model display device 1 includes a communication unit 11, a storage device 12, an imaging unit 13, a control unit 14, and a display unit 15. The 3D bone model display device 1 is a device that displays an augmented reality image in which a 3D bone model that resembles an animal's bone is projected into space as a virtual solid object. The 3D bone model display device 1 is realized by, for example, AR goggles or MR goggles. Note that the term "animal" refers to a living creature such as a person, dog, or cat, and in this embodiment, a case will be described in which a person (hereinafter referred to as a "patient") is the subject of the description.

[0017] The 3D bone model display device 1 is a device used by a surgeon who performs osteotomy on a deformed bone, for example. The 3D bone model is 3D data created to simulate the bone that is the target of the surgery.

[0018] The communication unit 11 is a communication circuit for receiving three-dimensional data representing a three-dimensional bone model that simulates the bones of a patient (i.e., an animal) from an external device. There are no particular limitations on the standard of communication performed by the communication unit 11. The external device is, for example, an information terminal that generates three-dimensional data representing the three-dimensional bone model from images of the patient's bones captured by an X-ray CT device or the like.

[0019] The communication unit 11 also stores the received three-dimensional data in the storage device 12 .

[0020] The storage device 12 is a memory or the like that stores the three-dimensional data, programs, etc. received by the communication unit 11 .

[0021] The imaging unit 13 is a camera or the like that captures an image of a part of the patient that has bone inside. The imaging unit 13 is attached to, for example, the frame of AR goggles or MR goggles. This allows the imaging unit 13 to capture an image of the field of view seen by the surgeon.

[0022] The control unit 14 controls the entire 3D bone model display device 1. The control unit 14 is realized by a microcomputer, a processor, or the like. That is, the functions of the control unit 14 are realized by the microcomputer, the processor, or the like executing a program stored in the storage device 12. The control unit 14 also includes a position identification unit 141 and an augmented reality image generation unit 142.

[0023] The position specifying unit 141 specifies the position of the bone in a part based on a part image obtained by the imaging unit 13 capturing an image of the part.

[0024] The augmented reality image generator 142 reads the three-dimensional data from the storage device 12 and generates and outputs an augmented reality image in which a three-dimensional bone model represented by the read three-dimensional data is positioned in a spatial position corresponding to the bone position identified by the position identification unit 141. For example, the augmented reality image generator 142 generates and outputs an augmented reality image in which the three-dimensional bone model is positioned so as to accurately overlay the patient's bone. More specifically, the augmented reality image generator 142 generates and outputs an augmented reality image in which the contour of the three-dimensional bone model matches the contour of the patient's bone in the direction of the surgeon's line of sight. Note that, in this specification, "generating and outputting an augmented reality image in which the contour of the three-dimensional bone model matches the contour of the patient's bone" means displaying, on the display unit 15, a three-dimensional bone model generated so that the contour of the three-dimensional bone model matches the contour of the real patient's bone. Furthermore, during the process in which the augmented reality image generator 142 generates the augmented reality image, the process of superimposing the three-dimensional bone model on the target patient's bone in the real surgical field is sometimes referred to as registration.

[0025] The augmented reality image generator 142 also generates and outputs an augmented reality image in which the 3D bone model follows the movement of the patient's body part. Specifically, when the spatial position of the bone identified by the position identification unit 141 changes, the augmented reality image generator 142 generates and outputs an augmented reality image in which the 3D bone model is positioned in a spatial position corresponding to the newly identified bone position. More specifically, the augmented reality image generator 142 generates and outputs an augmented reality image in which the outline of the 3D bone model matches the outline of the patient's bone in the direction of the surgeon's line of sight, as an image in which the 3D bone model follows the movement of the patient's body part. Note that, in this specification, "generating and outputting an augmented reality image in which the 3D bone model follows the movement of the patient's body part" means moving the position of the 3D bone model displayed on the display unit 15 in accordance with the movement of the patient's body part. The generation of an augmented reality image in which the 3D bone model follows the movement of the patient's body part by the augmented reality image generator 142 is sometimes referred to as tracking.

[0026] Furthermore, the augmented reality image generation unit 142 identifies the size of the bone in the local image captured by the imaging unit 13. When generating the augmented reality image, the augmented reality image generation unit 142 enlarges or reduces the three-dimensional bone model so that it matches the size of the bone and places it in space.

[0027] The display unit 15 is realized by, for example, the glasses (liquid crystal display) of AR goggles or MR goggles. The display unit 15 displays the augmented reality image output from the augmented reality image generation unit 142 as a three-dimensional image. The display unit 15 stereoscopically displays three-dimensional data representing a three-dimensional bone model as an augmented reality image. In other words, the parts and bones of the patient that the surgeon views via the display unit 15 are the parts and bones of the patient that the surgeon actually views.

[0028] [Identifying Bone Positions] The following is an example of a specific method by which the position identifying unit 141 identifies the positions of bones in a particular region based on an image of the region captured by the imaging unit 13. Note that the specific methods described below are merely examples, and the position identifying unit 141 may identify the positions of bones in a particular region using methods other than the specific methods described below.

[0029] The first method is a method using marking. Note that this method requires that at least one marking be made in advance at a specific position on the patient's bone. An example of the marking is a QR code (registered trademark).

[0030] First, when imaging a region having a bone therein, imaging unit 13 images the region so as to include the marking. Then, position specifying unit 141 specifies the position of the bone in the region image based on the position of the marking in the region image obtained by imaging unit 13. Position specifying unit 141 also specifies the size of the marking in the region image, thereby specifying the size of the bone in the region image.

[0031] The second method is based on image feature points.

[0032] First, the imaging unit 13 captures an image of a region containing a bone. Then, the position identification unit 141 extracts feature points from the image obtained by the imaging unit 13 capturing the image of the region. For example, the position identification unit 141 extracts bone contours, anatomical landmarks such as bony prominences, or individual-specific characteristic bone shapes from the image. When using bone contours, the position identification unit 141 matches the extracted bone contours with the contours of a three-dimensional bone model to identify the position of the bone in the region image. Furthermore, the position identification unit 141 identifies the size of the bone in the region image from the extracted bone contours.

[0033] According to the specific method described above, the position specifying unit 141 non-invasively aligns the patient's bones with the three-dimensional bone model, thereby reducing physical damage to the patient.

[0034] Furthermore, according to the above specific method, the augmented reality image generation unit 142 can enlarge or reduce the three-dimensional bone model and place it in space so that it corresponds to the position of the bone and matches the size of the bone.

[0035] [Operation] Next, the operation of the three-dimensional bone model display device 1 according to the present embodiment will be described with reference to Fig. 2. Fig. 2 is a flowchart showing the operation of the three-dimensional bone model display device 1 according to the embodiment.

[0036] First, the communication unit 11 receives three-dimensional data representing a three-dimensional bone model from an external device (step S1).

[0037] The communication unit 11 stores the three-dimensional data received in step S1 in the storage device 12 (step S2). That is, the storage device 12 holds the three-dimensional data stored in step S2.

[0038] The imaging unit 13 captures an image of the region having bone therein (step S3). That is, in step S3, the surgeon turns his or her face so that the region to be operated on is included in the surgical field (in other words, the field of view).

[0039] The position specifying unit 141 specifies the position of the bone in the region based on the region image obtained by the imaging unit 13 capturing an image of the region (step S4).

[0040] The augmented reality image generator 142 reads the three-dimensional data from the storage device 12, and generates and outputs an augmented reality image in which the three-dimensional bone model indicated by the read three-dimensional data is positioned in a space corresponding to the position of the bone identified in step S4 (step S5). More specifically, the augmented reality image generator 142 generates and outputs an augmented reality image in which the contour of the three-dimensional bone model coincides with the contour of the patient's bone in the direction of the surgeon's line of sight.

[0041] The display unit 15 displays the augmented reality image output by the augmented reality image generation unit 142 in step S5 (step S6).

[0042] Note that step S3 may be executed a certain period of time (e.g., one day) after step S2. For example, the 3D bone model display device 1 may execute steps up to S2 before surgery in response to an operation by the surgeon, and execute steps S3 and onward after the start of surgery.

[0043] Furthermore, during surgery, the three-dimensional bone model display device 1 can repeatedly execute steps S3 to S6 to display an augmented reality image in which the three-dimensional bone model follows the movement of the body part.

[0044] [Display Examples of Augmented Reality Images] Next, display examples of augmented reality images displayed on the display unit 15 will be described with reference to FIGS. 3 to 5. FIG.

[0045] FIG. 3 is a diagram showing an example of an augmented reality image displayed on the display unit 15 when the surgeon views the bone to be operated on from above (palmar side). That is, FIG. 3 is a diagram showing an example of the surgical field viewed by the surgeon. (a) of FIG. 3 is a diagram showing an example of the surgical field viewed by the surgeon before the display unit 15 displays the augmented reality image. (b) of FIG. 3 is a diagram showing an example of the augmented reality image viewed by the surgeon after the display unit 15 displays the augmented reality image. (c) of FIG. 3 is a diagram showing an example of the augmented reality image viewed by the surgeon following the movement of a site including the bone to be operated on during surgery, for example, after the augmented reality image is displayed on the display unit 15. (d) of FIG. 3 is a diagram showing an example of an augmented reality image in which an image of a 3D bone model after surgery is arranged in a space corresponding to the position of the bone. In the example shown in FIG. 3, the 3D bone model display device 1 displays 3D data indicating the 3D bone model and the fixation members described later as an augmented reality image. In other words, the part of the patient (the arm in the example shown in Fig. 3) and the bones in that part are the part of the patient and the bones in that part that the surgeon actually sees. Also, in Fig. 3(b) to (d), for ease of explanation, the bone to be operated on and the 3D bone model are drawn shifted, but in reality, an augmented reality image in which the outline of the bone to be operated on and the outline of the 3D bone model match is displayed on the display unit 15.

[0046] The diagram shown in (a) of Figure 3 shows the surgical field after the surgeon has made an incision in the area containing the deformed bone (in the example shown in Figure 3, the patient's arm) and exposed the bone.

[0047] The 3D bone model display device 1 displays an augmented reality image during surgery, in which the 3D bone model is positioned in space corresponding to the position of the bone, as shown in Figure 3(b). In other words, the image shown in Figure 3(b) is an image obtained by registration. The rectangular planar figure attached to the 3D bone model indicates the plane along which the surgeon will perform osteotomy during surgery. The shape of the figure attached to the 3D bone model is not particularly limited, as long as it indicates the position or direction of manipulation during surgery.

[0048] When a part including the target bone moves during the surgical procedure (i.e., when its spatial position is changed), the 3D bone model display device 1 displays an augmented reality image during the surgical procedure in which the 3D bone model tracks the position of the bone, as shown in (c) of Figure 3. In other words, the image shown in (c) of Figure 3 is an image obtained by tracking. Furthermore, the rectangular planar figure attached to the 3D bone model indicates the plane along which the surgeon will perform osteotomy during the surgical procedure. The shape of the figure attached to the 3D bone model is not particularly limited, as long as it indicates the position or direction of surgical manipulation.

[0049] Furthermore, the 3D bone model display device 1 also performs tracking when, for example, the surgeon changes the direction of his or her face. That is, the 3D bone model display device 1 performs tracking when the position of the bone to be operated on moves in the local image obtained by the imaging unit 13.

[0050] The surgeon views an augmented reality image of the post-operative 3D bone model, which is placed in a space corresponding to the position of the bone, as shown in (d) of Figure 3. The 3D bone model display device 1 displays, during surgery, an image of the post-operative state in which, for example, a fixation member has been installed to connect two bones separated by osteotomy, allowing the surgeon to compare the image with the bone during surgery.

[0051] 3, the 3D bone model and fixing members, which are virtual solid objects, are preferably displayed semi-transparently to the extent that the patient's parts and bones can be seen through them. In other words, the 3D bone model display device 1 preferably displays the 3D bone model and fixing members semi-transparently so as not to obstruct the surgeon's visibility of the patient's parts and bones superimposed on the 3D bone model and fixing members.

[0052] Furthermore, in response to an operation by the user, the 3D bone model display device 1 may switch the augmented reality image to be displayed. For example, the 3D bone model display device 1 may display the augmented reality image shown in Fig. 3(b) after displaying the augmented reality image shown in Fig. 3(d).

[0053] Next, a method for checking the accuracy of registration of the augmented reality image displayed on the display unit 15 will be described with reference to Fig. 4. Fig. 4 is a diagram showing a scene in which the augmented reality image shown in Fig. 3(b) is checked.

[0054] By comparing the augmented reality image shown in Figure 4 with an X-ray image of the deformed bone, it can be confirmed that the augmented reality image displayed by the 3D bone model display device 1 is accurately superimposed on the bone in the actual surgical field.

[0055] Next, an augmented reality image displayed on the display unit 15 when the surgeon views the bone to be operated on from the side when the augmented reality image shown in FIG. 3B or 3C is displayed will be described with reference to FIG. 5 . FIG. 5 is a diagram showing an example of an augmented reality image displayed on the display unit 15 when the surgeon views the bone to be operated on from the side (radial side). That is, FIG. 5 is a diagram showing an example of the surgical field viewed by the surgeon. Note that in FIG. 5 , the bone to be operated on and the three-dimensional bone model are depicted offset from each other for ease of explanation, but in reality, an augmented reality image in which the outline of the bone to be operated on and the outline of the three-dimensional bone model coincide is displayed on the display unit 15.

[0056] 5, the 3D bone model display device 1 displays an augmented reality image during surgery in which a 3D bone model is accurately superimposed on the bone. As a result, the 3D bone model display device 1 displays an augmented reality image that corresponds to the spatial position of the actual bone, so that the augmented reality image can be displayed during surgery in a form that is easier for the surgeon to refer to than conventional technology in which an augmented reality image is displayed on a display positioned at a position unrelated to the spatial position of the actual bone.

[0057] [Effects] The three-dimensional bone model display device 1 according to this embodiment displays an augmented reality image in which a three-dimensional bone model modeled on an animal's bone is projected onto space as a virtual solid object, and includes: a storage device 12 that stores three-dimensional data representing the three-dimensional bone model; an imaging unit 13 that captures an image of a part of the animal that has bone inside; a position identification unit 141 that identifies the position of the bone in the part based on the part image obtained by imaging the part with the imaging unit 13; an augmented reality image generation unit 142 that reads the three-dimensional data from the storage device 12, and generates and outputs an augmented reality image in which the three-dimensional bone model indicated by the read three-dimensional data is positioned in a position in space corresponding to the position of the bone identified by the position identification unit 141; and a display unit 15 that displays the augmented reality image output from the augmented reality image generation unit 142.

[0058] The 3D bone model display device 1 displays an augmented reality image in which a 3D bone model is positioned in a spatial position corresponding to the position of the animal's bones. Therefore, unlike conventional technology that displays an augmented reality image on a display, the surgeon can perform surgery while comparing the animal's bones with the 3D bone model in the same field of view. This allows the surgeon to accurately recognize the position where the bones will be manipulated, so the 3D bone model display device 1 can provide the surgeon with more thorough support during surgery.

[0059] Furthermore, in the three-dimensional bone model display device 1 according to this embodiment, the three-dimensional bone model display device 1 targets a bone having at least one marking, the imaging unit 13 images the part including the marking, and the position identification unit 141 identifies the position of the bone in the part based on the marking included in the part image obtained by imaging the part with the imaging unit 13.

[0060] The position identification unit 141 identifies the position of the bone based on the markings included in the local image, so the augmented reality image generation unit 142 can reliably place the three-dimensional bone model in space corresponding to the position of the bone.

[0061] Furthermore, in the three-dimensional bone model display device 1 according to this embodiment, the position identification unit 141 extracts feature points from the part image obtained by the imaging unit 13 capturing an image of the part, and identifies the position of the bone in the part based on the extracted feature points.

[0062] The position identification unit 141 identifies the position of the bone based on the extracted feature points, so the augmented reality image generation unit 142 can place the three-dimensional bone model in space corresponding to the position of the bone without marking the bone or the like.

[0063] Furthermore, in the three-dimensional bone model display device 1 according to this embodiment, the augmented reality image generation unit 142 further generates and outputs an augmented reality image in which the three-dimensional bone model follows the movement of the body part.

[0064] The augmented reality image generator 142 generates and outputs an augmented reality image that tracks the 3D bone model as the part moves, so even if the part moves during surgery, the position where the bone is to be manipulated can be accurately recognized. This allows the 3D bone model display device 1 to provide continuous and thorough support to the surgeon during surgery, even if the part moves.

[0065] Furthermore, the three-dimensional bone model display method according to this embodiment is a three-dimensional bone model display method executed by a three-dimensional bone model display device 1 that displays an augmented reality image in which a three-dimensional bone model model modeling an animal's bone is projected onto space as a virtual solid object. The three-dimensional bone model display device 1 includes a storage device 12 that stores three-dimensional data representing the three-dimensional bone model, and an imaging unit 13 that images a part of the animal that has bone inside. The three-dimensional bone model display method includes a position identification step (S4) that identifies the position of the bone in the part based on a part image obtained by imaging the part with the imaging unit 13, an augmented reality image generation step (S5) that reads the three-dimensional data from the storage device 12, and generates and outputs an augmented reality image in which the three-dimensional bone model represented by the read three-dimensional data is positioned in space at a position corresponding to the position of the bone identified in the position identification step (S4), and a display step (S6) that displays the augmented reality image output in the augmented reality image generation step (S5).

[0066] The 3D bone model display method displays an augmented reality image in which a 3D bone model is positioned in a spatial position corresponding to the position of the animal's bone. Therefore, unlike conventional techniques that display augmented reality images on a display, the surgeon can perform surgery while comparing the animal's bones with the 3D bone model in the same field of view. This allows the surgeon to accurately recognize the position where the bones should be manipulated, so the 3D bone model display method can provide the surgeon with more thorough support during surgery.

[0067] The program according to the present embodiment causes a computer to execute the three-dimensional bone model display method according to the present embodiment.

[0068] Such a program provides the same effects as the three-dimensional bone model display method according to the present embodiment.

[0069] [Modifications] The 3D bone model display device and the like according to the present disclosure have been described above based on the above embodiment, but are not limited to the above embodiment. As long as they do not deviate from the spirit of the present disclosure, various modifications conceivable by those skilled in the art to the above embodiment and forms constructed by combining components of different embodiments may also be included within the scope of one or more aspects.

[0070] In the above embodiments, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0071] In the above-described embodiments, some or all of the functions of the components may be realized by a processor such as a CPU executing a program.

[0072] Furthermore, the method of communication between the devices in the above-described embodiment is not particularly limited, and a relay device (not shown) may be involved in the communication between the devices.

[0073] In the above-described embodiment, the processing performed by a specific processing unit may be performed by another processing unit. The order of multiple processing operations may be changed, or multiple processing operations may be performed in parallel.

[0074] Furthermore, some or all of the components constituting each of the above devices may be configured as an IC card or a standalone module that can be attached to or detached from each device. The IC card or module is a computer system configured from a microprocessor, ROM, RAM, etc. The IC card or module may include a super multi-function LSI. The IC card or module achieves its functions when the microprocessor operates in accordance with a computer program. The IC card or module may be tamper-resistant.

[0075] The present disclosure may also be realized as a method executed by a computer, or as a program for causing a computer to execute the method. The present disclosure may also be realized as a computer-readable non-transitory recording medium on which such a program is recorded. Such a program includes an application program for causing a general-purpose information terminal (computer) to function as the setting terminal of the above-described embodiment, and an application program installed on the information terminal.

[0076] A three-dimensional bone model display device according to the present disclosure is useful, for example, as a surgical assistance tool during surgery.

[0077] REFERENCE SIGNS LIST 1 3D bone model display device 11 Communication unit 12 Storage device 13 Imaging unit 14 Control unit 141 Position identification unit 142 Augmented reality image generation unit 15 Display unit

Claims

1. A three-dimensional bone model display device that displays an augmented reality image in which a three-dimensional bone model modeling an animal's bone is projected onto space as a virtual solid object, comprising: a storage device that stores three-dimensional data representing the three-dimensional bone model; an imaging unit that captures images of parts of the animal that have the bone inside; a position identification unit that identifies the position of the bone in the part based on the part image obtained by the imaging unit capturing the part; an augmented reality image generation unit that reads the three-dimensional data from the storage device, and generates and outputs an augmented reality image in which the three-dimensional bone model represented by the read three-dimensional data is positioned in a position in space corresponding to the position of the bone identified by the position identification unit; and a display unit that displays the augmented reality image output from the augmented reality image generation unit.

2. The three-dimensional bone model display device according to claim 1, wherein the three-dimensional bone model display device targets the bone having at least one marking, the imaging unit images the part including the marking, and the position identification unit identifies the position of the bone in the part based on the marking included in the part image obtained by the imaging unit imaging the part.

3. The three-dimensional bone model display device according to claim 1, wherein the position identification unit extracts feature points from an image of the part obtained by the imaging unit capturing an image of the part, and identifies the position of the bone in the part based on the extracted feature points.

4. The 3D bone model display device according to any one of claims 1 to 3, wherein the augmented reality image generation unit further generates and outputs an augmented reality image in which the 3D bone model follows the movement of the part.

5. A three-dimensional bone model display method executed by a three-dimensional bone model display device that displays an augmented reality image in which a three-dimensional bone model model modeling an animal's bone is projected into space as a virtual solid object, wherein the three-dimensional bone model display device comprises: a storage device that stores three-dimensional data representing the three-dimensional bone model; and an imaging unit that images a part of the animal that has the bone inside, the three-dimensional bone model display method comprising: a position identification step that identifies the position of the bone in the part based on a part image obtained by the imaging unit imaging the part; an augmented reality image generation step that reads the three-dimensional data from the storage device, and generates and outputs an augmented reality image in which the three-dimensional bone model represented by the read three-dimensional data is positioned in a position in space corresponding to the position of the bone identified in the position identification step; and a display step that displays the augmented reality image output in the augmented reality image generation step.

6. A program for causing a computer to execute the three-dimensional bone model display method according to claim 5.

Images