Program, information processing method, and information processing device

Abstract

To enable superimposition, for display, of an inner plate line indicating the maximum shaving point when shaving a pelvis to place a cup included in an artificial hip joint.SOLUTION: Provided is a program that causes a computer to execute a process for: acquiring a fluoroscopic image of a patient who is to undergo treatment pertaining to an artificial hip joint; recognizing, in the pelvis of the patient in the acquired fluoroscopic image, an inner plate line indicating a maximum shaving point during pelvic shaving in order to arrange a cup included in the artificial hip joint; recognizing, in the patient's pelvis, a must-reach line indicating a minimum shaving point during pelvic shaving in order to arrange the cup; deriving a cup CE angle for when arrangement of the cup is assumed in a state where the patient's pelvis has been resected with a bone reamer used in the treatment pertaining to the artificial hip joint; displaying the recognized inner plate line and must-reach line superimposed onto the fluoroscopic image; and displaying the derived cup CE angle in conjunction with the fluoroscopic image.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a program, an information processing method, and an information processing device. [Background technology]

[0002] An acetabular socket angle setting device for use in total hip replacement surgery is known (for example, Patent Document 1). The acetabular socket angle setting device for use in total hip replacement surgery described in Patent Document 1 provides a method for setting an acetabular socket angle based on the pelvic surface and the inter-tear line. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-031047 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the acetabular socket angle setting device for total hip replacement surgery in Document 1 does not take into consideration the need to superimpose an inner plate line, indicating the maximum cutting point when cutting the pelvis to place the cup included in the artificial hip joint, on an X-ray image of a patient undergoing treatment related to the artificial hip joint.

[0005] The present disclosure has been made in consideration of the above circumstances, and aims to provide a program or the like that can superimpose an inner plate line, indicating the maximum point of reduction when reducing the pelvis to position a cup included in the artificial hip joint, on an X-ray image of a patient undergoing treatment related to an artificial hip joint. [Means for solving the problem]

[0006] In one proposal, the program causes a computer to execute a process of acquiring an fluoroscopy image of a patient undergoing a procedure related to an artificial hip joint, recognizing an inner plate line in the patient's pelvis in the acquired fluoroscopy image, which indicates the maximum point at which the pelvis must be cut to place a cup included in the artificial hip joint, recognizing a required line in the patient's pelvis, which indicates the minimum point at which the pelvis must be cut to place the cup, deriving a cup CE angle when it is assumed that the cup will be placed in a state in which the patient's pelvis has been cut using a bone drilling reamer used in the procedure related to the artificial hip joint, superimposing the recognized inner plate line and required line on the fluoroscopy image, and displaying the derived cup CE angle in association with the fluoroscopy image.

[0007] In one proposal, an information processing method instructs a computer to execute a process of acquiring an fluoroscopy image of a patient undergoing a procedure related to an artificial hip joint, recognizing an inner plate line in the patient's pelvis in the acquired fluoroscopy image, which indicates the maximum point of cutting when the pelvis is cut to place a cup included in the artificial hip joint, recognizing a required line in the patient's pelvis, which indicates the minimum point of cutting when the pelvis is cut to place the cup, deriving a cup CE angle when it is assumed that the cup will be placed in a state in which the patient's pelvis has been cut using a bone drilling reamer used in the procedure related to the artificial hip joint, superimposing the recognized inner plate line and required line on the fluoroscopy image, and displaying the derived cup CE angle in association with the fluoroscopy image.

[0008] In one proposal, the information processing device comprises an acquisition unit that acquires an fluoroscopy image of a patient undergoing a procedure related to an artificial hip joint, an inner plate line recognition unit that recognizes an inner plate line in the patient's pelvis in the acquired fluoroscopy image, which indicates the maximum cutting point when the pelvis is cut to place a cup included in the artificial hip joint, a must-reach line recognition unit that recognizes a must-reach line in the patient's pelvis, which indicates the minimum cutting point when the pelvis is cut to place the cup, a derivation unit that derives a cup CE angle when it is assumed that the cup will be placed in a state in which the patient's pelvis has been cut with a bone-drilling reamer used in the procedure related to the artificial hip joint, and a display unit that superimposes the recognized inner plate line and the must-reach line on the fluoroscopy image and displays the derived cup CE angle in association with the fluoroscopy image. Effect of the Invention

[0009] According to the present disclosure, it is possible to superimpose an inner plate line indicating the maximum point of reduction when reducing the pelvis to position the cup included in the artificial hip joint on a fluoroscopy image of a patient undergoing a procedure related to an artificial hip joint. [Brief description of the drawings]

[0010] [Figure 1] 1 is a schematic diagram showing an overview of an intraoperative support system including an information processing device according to a first embodiment. [Diagram 2] 1 is a block diagram showing an example of the configuration of an information processing device; [Diagram 3] FIG. 2 is an explanatory diagram showing an example of a learning model (inner plate line model). [Figure 4] 11 is a flowchart showing an example of a processing procedure (during model learning) of a processing unit of the information processing device. [Diagram 5] 13 is a flowchart showing an example of a processing procedure (during model operation) of a processing unit of the information processing device. [Figure 6] FIG. 1 is an explanatory diagram regarding a cup CE angle. [Figure 7] FIG. [Figure 8] FIG. 13 is an explanatory diagram of a required line. [Figure 9] FIG. 13 is an explanatory diagram regarding an upward limit line. [Figure 10] FIG. 2 is an explanatory diagram regarding the strike zone. [Figure 11] FIG. 13 is an explanatory diagram of the anteversion angle and abduction angle of the cup (assumed position after the pelvis has been removed with a bone-drilling reamer). [Figure 12] FIG. 13 is an explanatory diagram illustrating an example of a display screen of support information (support information superimposed screen). [Figure 13] FIG. 13 is an explanatory diagram illustrating an example of a display screen (distortion correction screen) for support information. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The present invention will now be described in detail with reference to the drawings showing embodiments thereof. (Embodiment 1)

[0012] FIG. 1 is a schematic diagram showing an overview of an intraoperative support system S including an information processing device 1 according to the first embodiment. FIG. 2 is a block diagram showing an example of the configuration of the information processing device 1. The intraoperative support system S is configured with the information processing device 1 as a main device, and a fluoroscopic image capturing device such as an X-ray device 62 and a tomographic image capturing device such as a CT device 61 are communicably connected to the information processing device 1. The information processing device 1 acquires X-ray images (fluoroscopic image capturing) captured by the X-ray device 62 in real time. The information processing device 1 acquires CT images (tomographic images) captured by the CT device 61. The information processing device 1 may further be communicably connected to an electronic medical record server that stores and manages various medical data related to a patient K.

[0013] A CT image (tomographic image) of a patient K undergoing treatment for an artificial hip joint 8 is taken by a CT device 61 before (preoperatively) the pelvis is excavated by the bone excavation reamer 7. An X-ray image (fluoroscopic image) of a patient K undergoing treatment for an artificial hip joint 8 is taken by an X-ray device 62 while (intraoperatively) the pelvis is being excavated by the bone excavation reamer 7. Therefore, the X-ray image includes the bone excavation reamer 7 excavating the pelvis (during excavation).

[0014] After a doctor has appropriately excavated the pelvis using a bone-excavating reamer 7, an artificial hip joint 8 including a cup 81 is inserted (implanted) into the body of patient K. The artificial hip joint 8 includes a hemispherical cup 81 that is fitted to the excavated pelvis, and a stem that is inserted into the femur. The bone-excavating reamer 7 has a hemispherical tip portion for excavation, and the outer edge shape of the tip portion is the same shape and size as the outer edge shape of the cup 81 of the artificial hip joint 8.

[0015] Although details will be described later, the processing unit 2 of the information processing device 1 performs shape recognition on the bone drilling reamer 7 (semispherical tip portion performing drilling) included in the acquired X-ray image using, for example, an object detection model such as YOLO or edge detection, and identifies the position and inclination of the bone drilling reamer 7 on the pelvis. The processing unit 2 of the information processing device 1 regards the identified position and inclination of the bone drilling reamer 7 as the position and inclination of the cup 81 assumed to be disposed in a state in which the pelvis is scraped by the bone drilling reamer 7, and calculates and outputs various information related to the cup 81 assumed to be disposed. Furthermore, the processing unit 2 of the information processing device 1 superimposes an inner plate line (reach area) on the X-ray image to support drilling by the bone drilling reamer 7, using inner plate corresponding points (landmarks) derived using a learning model 101 (inner plate line model) for multiple CT images captured before surgery.

[0016] The information processing device 1 is a computer capable of various information processing and sending and receiving information, such as a server device or a personal computer. The server device includes not only a single server device, but also a cloud server device or a virtual server device configured with multiple computers. When the information processing device 1 is configured as a cloud server device, for example, the information processing device 1 does not need to be installed in a medical facility where the patient K is located, like medical devices such as a CT device 61 or an X-ray device 62, and may be connected to these medical devices via an external network such as the Internet so as to be able to communicate with them. The information processing device 1 includes a processing unit 2, a storage unit 3, an input / output I / F 4, and a communication unit 5.

[0017] The processing unit 2 has an arithmetic processing device equipped with a timing function such as one or more CPUs (Central Processing Units), MPUs (Micro-Processing Units), GPUs (Graphics Processing Units), etc., and performs various information processing, control processing, etc. related to the information processing device 1 by reading and executing a program P (program product) stored in the memory unit 3.

[0018] The storage unit 3 includes a volatile storage area such as a static random access memory (SRAM), a dynamic random access memory (DRAM), a flash memory, etc., and a non-volatile storage area such as an EEPROM or a hard disk. The storage unit 3 stores in advance a program P (program product) and data to be referenced during processing. The program P (program product) stored in the storage unit 3 may be a program P (program product) read from a recording medium M readable by the information processing device 1. Alternatively, the program P (program product) may be downloaded from an external computer (not shown) connected to a communication network (not shown) and stored in the storage unit 3.

[0019] The storage unit 3 stores various medical data related to the patient K and various values ​​determined in the preoperative plan. The storage unit 3 further stores an entity file constituting a learning model 101 (inner plate line model). The entity file may be configured as a part of a program P (program product).

[0020] The communication unit 5 is a communication module or communication interface for communicating with an electronic medical record server or an information terminal such as a smartphone held by a medical professional by wired or wireless means, for example, a wired communication module such as an Ethernet (registered trademark) connector, a narrow-range wireless communication module such as WiFi (registered trademark) or Bluetooth (registered trademark), or a wide-range wireless communication module such as 4G or 5G. The processing unit 2 communicates with the electronic medical record server or the information terminal via the communication unit 5, for example, via a local network within the medical institution or an external network such as the Internet.

[0021] The input / output I / F 4 is a communication interface that complies with a communication standard such as RS232C or USB. An input device such as a keyboard or a display device 41 such as a liquid crystal display is connected to the input / output I / F 4. Furthermore, the input / output I / F 4 may be connected to medical devices such as a CT device 61 or an X-ray device 62.

[0022] 3 is an explanatory diagram showing an example of a learning model 101 (inner plate line model). The learning model 101 (inner plate line model) is a neural network (NN) that is configured, for example, by RCNN (Regions with Convolutional Neural Network), Fast RCNN, Faster RCNN, SSD (Single Shot Multibook Detector), YOLO (You Only Look Once), etc., and performs object detection, semantic segmentation, or instance segmentation.

[0023] The learning model 101 (inner plate line model) outputs whether or not an inner plate corresponding point is included (presence) based on an input image (a tomographic image such as a CT image), and if an inner plate corresponding point is included (presence), outputs the area (position information) of the inner plate corresponding point in the input image. In other words, the learning model 101 functions as an area extraction model that extracts the area of ​​the inner plate corresponding point included in the input image (a tomographic image such as a CT image).

[0024] When the learning model 101 (inner plate line model) is configured with a neural network including a CNN (Convolutional Neural Network) that extracts image features, such as RCNN, the input layer included in the learning model 101 (inner plate line model) has a plurality of neurons that accept input of pixel values ​​of an image, and passes the input pixel values ​​to the intermediate layer. The intermediate layer has a plurality of neurons that extract image features of the image, and passes the extracted image features to the output layer. The output layer has one or a plurality of neurons that output area information including the positions of the inner plate corresponding points, and outputs the positions of the inner plate corresponding points (area coordinates or pixel numbers, etc.) based on the image features output from the intermediate layer.

[0025] The neural network (learning model 101) trained using the training data is expected to be used as a program module that is part of artificial intelligence software. The learning model 101 is used in the information processing device 1 equipped with the processing unit 2 (CPU, etc.) and the storage unit 3 as described above, and is executed in the information processing device 1 having such calculation processing capabilities to form a neural network system. That is, the processing unit 2 of the information processing device 1 performs calculations to extract the feature amount of the image input to the input layer according to the command from the learning model 101 stored in the storage unit 3, and outputs the position (area) of the inner plate corresponding point from the output layer.

[0026] The learning model 101 (internal plate line model) can be generated by preparing training data in which a tomographic image (question data) such as a CT image including the pelvis (hip joint) is associated with a label (answer data) indicated by annotating the position (area) of the internal plate corresponding point, which is a point on the internal plate line in the pelvis, and by machine learning an unlearned neural network using the training data. The training data is stored, for example, in the storage unit 3 of the information processing device 1, and can be generated by aggregating images and doctor's findings contained in a large number of diagnosis results or surgery results stored in an electronic medical record server or the like in a medical institution such as a hospital. That is, the internal plate line in the pelvis (the internal plate corresponding point, which is a point on the internal plate line) is a location (internal body part) identified based on the findings of a doctor or the like. According to the learning model 101 (internal plate line model) thus learned and configured, by inputting a tomographic image such as a CT image into the learning model 101, information indicating the position (area coordinate in the image coordinate system) of the internal plate corresponding point included in the tomographic image can be obtained.

[0027] In this embodiment, the learning model 101 (inner plate line model) is described as being RCNN, but the learning model 101 is not limited to RCNN, and may be a learning model 101 constructed with other learning algorithms, such as a neural network other than RCNN, a Support Vector Machine (SVM), a transformer, YOLO, a Bayesian network, a regression tree, etc. The data set of question data and answer data included in the training data for learning the learning model 101 and the data set of input data and output data when using the learning model 101 are synonymous, and if it is defined in one data set, it naturally applies to the other data set.

[0028] 4 is a flowchart showing an example of a processing procedure (during model learning) of the processing unit 2 of the information processing device 1. The processing unit 2 of the information processing device 1 receives an operation by an operator using, for example, a keyboard connected to the input / output I / F 4, and performs the following processing based on the received operation.

[0029] The processing unit 2 of the information processing device 1 acquires training data (S11). The processing unit 2 of the information processing device 1 acquires training data to which position information of inner plate corresponding points (landmarks) indicating points on the inner plate line is added for a tomographic image (CT image, etc.). The position information of the inner plate corresponding points (landmarks) may be, for example, a figure such as a circle annotated at a location (pelvis area) indicating the inner plate corresponding point for the tomographic image.

[0030] The processing unit 2 of the information processing device 1 uses the acquired training data to generate a learning model 101 (inner plate line model) (S12). The processing unit 2 of the information processing device 1 inputs training data (an image in which the tomographic image is annotated at the locations indicating the inner plate corresponding points) to an untrained neural network, thereby training the neural network and generating the learning model 101 (inner plate line model).

[0031] In this embodiment, the learning model 101 outputs position information of the inner plate corresponding points (landmarks) indicating points on the inner plate line based on the input CT image (superimposing the inner plate corresponding points (landmarks) on the CT image), but is not limited to this. The learning model 101 may also output, in addition to the inner plate corresponding points (landmarks) indicating points on the inner plate line, the required corresponding points indicating points on the required line described later, and the upward limit point indicating points on the upward limit line, based on the input CT image. In this case, the learning model 101 may be input with a target CE angle in addition to the CT image. That is, the learning model 101 may function as a reachable area model that outputs points on the inner plate line, the required line, and the upward limit line (inner plate corresponding points, required corresponding points, and upward limit points) for identifying the reachable area based on the input CT image (or the CT image and the target CE angle). In this case, the reachable region model is trained using training data including answer data in which inner plate corresponding points, must-reach corresponding points, and upper limit points are annotated for the CT image, which is the problem data.

[0032] 5 is a flowchart showing an example of a processing procedure (during model operation) of the processing unit 2 of the information processing device 1. The processing unit 2 of the information processing device 1 starts the processing of this flowchart, for example, when a fluoroscopic image is input as a trigger, or when a start command or the like is received from an input device such as a keyboard connected to the input / output I / F 4.

[0033] The processing unit 2 of the information processing device 1 acquires a tomographic image (CT image) of the pelvis of the patient K (S101). Before the drilling treatment by the bone drilling reamer 7 (preoperatively), the patient K has had a tomographic image (CT image or MRI image) of the internal body part including the pelvis and hip joint taken by, for example, a tomographic image taking device of the CT device 61 or an MRI device. The tomographic image taken before the operation is stored, for example, in the storage unit 3 of the information processing device 1. Alternatively, these tomographic images (CT images, etc.) may be stored in an electronic medical record server, and the processing unit 2 of the information processing device 1 may access the electronic medical record server by, for example, a patient KID that uniquely identifies the patient K, and acquire the tomographic image (CT image, etc.) from the electronic medical record server. The tomographic image includes a plurality of tomographic images sliced ​​at a predetermined slice width (up and down (Y axis) direction of the human body). That is, in this embodiment, the Y axis indicates the upper limit direction of the human body, and the positive direction of the Y axis indicates the upward direction. The X-axis indicates the left-right direction of the human body (the positive direction is to the right on the paper), and the Z-axis indicates the front-back direction of the human body (the positive direction is forward).

[0034] The processing unit 2 of the information processing device 1 inputs the tomographic image to the learning model 101 (inner plate line model) (S102). The processing unit 2 of the information processing device 1 inputs each of the acquired multiple tomographic images to the learning model 101 (inner plate line model). The learning model 101 (inner plate line model) has been trained to output position information (landmarks) of the inner plate (inner plate corresponding points on the inner plate line) when a tomographic image is input. The learning model 101 (inner plate line model) outputs the position information (landmarks) of the inner plate by superimposing it on the input tomographic image based on the input tomographic image.

[0035] The processing unit 2 of the information processing device 1 acquires inner plate corresponding points (landmarks) indicating points on the inner plate line from the learning model 101 (S103). The processing unit 2 of the information processing device 1 acquires, from the learning model 101, a tomographic image on which the inner plate corresponding points (landmarks) indicating points on the inner plate line are superimposed, and stores the acquired multiple tomographic images (tomographic images on which the inner plate corresponding points are superimposed) in the storage unit 3.

[0036] The processing unit 2 of the information processing device 1 acquires various values ​​determined in the preoperative plan by referring to the storage unit 3 (S104). The various values ​​determined in the preoperative plan include, for example, the radius of the cup 81 of the artificial hip joint 8 to be placed inside the body, the target CE angle, the target distance between the outer edge of the cup 81 and the inner plate line, the target anteversion angle, and the target abduction angle, which are preset as target values.

[0037] The target CE angle indicates the cup CE angle when the cup 81 is placed, which is preset as a target value. The target distance indicates the distance between the cup 81, which is preset as a target value, and the inner plate line derived using the learning model 101 (inner plate line model) based on the tomographic image. The target anterior inclination angle indicates the anteversion angle of the cup 81 when the cup 81 is placed, which is preset as a target value. The target abduction angle indicates the abduction angle of the cup 81 when the cup 81 is placed, which is preset as a target value. These various values ​​determined in the preoperative plan are determined by a doctor or the like, inputted in advance to the information processing device 1, and stored in the storage unit 3. Alternatively, the processing unit 2 of the information processing device 1 may acquire various values ​​determined in the preoperative plan for the patient K from an electronic medical record system based on the patient KID or the like.

[0038] The processing unit 2 of the information processing device 1 acquires a fluoroscopic image (X-ray image) of the pelvis of the patient K during surgery for the patient K undergoing treatment for the artificial hip joint 8 (S105). The processing unit 2 of the information processing device 1 acquires a fluoroscopic image (X-ray image) of the pelvis of the patient K in real time during surgery for the patient K undergoing treatment for the artificial hip joint 8, that is, in a state in which the pelvis is being excavated by the bone excavation reamer 7. The processing unit 2 of the information processing device 1 may acquire the fluoroscopic image (X-ray image) in real time in the form of a moving image. In this way, the fluoroscopic image (X-ray image) of the patient K acquired in real time during surgery includes the bone excavation reamer 7 that cuts the pelvis, and the current position and inclination of the bone excavation reamer 7 (the outer edge of the hemispherical tip that performs excavation) in the pelvis can be specified. When the placement of cup 81 is assumed to be in the current position of the bone-drilling reamer 7 in the pelvis, i.e., in the bone depression (installation space) excavated by the bone-drilling reamer 7, the cup CE angle, anteversion angle, and abduction angle of the assumedly positioned cup 81 can be derived.

[0039] 6 is an explanatory diagram of the cup CE angle. In the illustration of this embodiment, a semicircular graphic object showing a cup 81 assumed to be placed based on the current position of the bone drilling reamer 7 in the pelvis is displayed superimposed on a perspective image. The processing unit 2 of the information processing device 1 derives, as the cup CE angle, an angle formed by a line connecting the center (COR: Center of Rotation) of the cup 81 assumed to be placed in a state in which the pelvis is scraped by the bone drilling reamer 7 to the intersection point of the outer edge of the cup 81 and the white line (acetabular sourcil) of the pelvis load surface, and a line connecting the lower ends of the left and right teardrops in the pelvis, which is perpendicular to the pelvis reference line (Y axis).

[0040] The processing unit 2 of the information processing device 1 derives an inner plate line in the perspective image based on the inner plate corresponding points (landmarks) output by the learning model 101 (S106). The learning model 101 outputs a tomographic image in which the inner plate corresponding points (landmarks) are superimposed on the input tomographic image (CT image, etc.). The inner plate corresponding points are points located on the inner plate line indicating the maximum cutting point when cutting the pelvis, that is, the excavation limit line that must not be exceeded. The processing unit 2 of the information processing device 1 extracts each of the points (landmarks) shown in each tomographic image and forms (connects) a line connecting these extracted points (landmarks), thereby deriving (identifying) the inner plate line.

[0041] FIG. 7 is an explanatory diagram of the inner plate line. In the illustration of this embodiment, a plurality of (three in the embodiment) tomographic images are shown, and each tomographic image has a correspondence relationship (same Y coordinate) with a perspective image (X-ray image) at a plurality of points in the Y-axis direction (up and down direction of the human body). Each of these tomographic images is output from the learning model 101 (inner plate line model), and a circle indicating an inner plate corresponding point (landmark) is superimposed on each tomographic image. The processing unit 2 of the information processing device 1 uses each of the tomographic images on which the inner plate corresponding points (landmarks) are superimposed in this way to identify a plurality of inner plate corresponding points (landmarks) in the perspective image, and forms an inner plate line by connecting the identified plurality of inner plate corresponding points.

[0042] The processing unit 2 of the information processing device 1 may generate (reconstruct) a three-dimensional medical image including the pelvis and hip joint of the patient K from volume data constructed using these multiple tomographic images. In addition, the processing unit 2 of the information processing device 1 may set a three-dimensional internal coordinate system in the tomographic image or three-dimensional medical image including the pelvis, etc. of the patient K. In the internal coordinate system, the processing unit 2 of the information processing device 1 may define, for example, a line connecting the lower ends of two teardrops of the pelvis as a pelvic reference line, define a horizontal direction (left-right direction with respect to the human body) with respect to the pelvic reference line as the X-axis, a vertical direction (up-down direction with respect to the human body) with respect to the pelvic reference line as the Y-axis, and a direction perpendicular to both the X-axis and the Y-axis (front-back direction with respect to the human body) as the X-axis, and perform various arithmetic processing. The processing unit 2 of the information processing device 1 may normalize the internal body coordinate system between the obtained fluoroscopic image (X-ray image) and the obtained tomographic image (CT image), that is, align them, using the shape recognition results of the same internal body part (e.g., tear stain, etc.) included in both the obtained fluoroscopic image (X-ray image) and the obtained tomographic image (CT image). As a result, the processing unit 2 of the information processing device 1 applies the internal body coordinate system set in the multiple tomographic images (or reconstructed 3D medical images) to the fluoroscopic image (X-ray image) captured in real time during surgery, and derives an internal plate line that connects multiple internal plate corresponding points in the fluoroscopic image (X-ray image) based on the internal plate corresponding points (landmarks) identified in each tomographic image. Furthermore, the processing unit 2 of the information processing device 1 can also identify the current position of the bone drilling reamer 7 included in the fluoroscopic image (X-ray image) in the internal body coordinate system that is a common coordinate system with the tomographic image (CT image) by converting from the image coordinate system of the fluoroscopic image to the internal body coordinate system.

[0043] The processing unit 2 of the information processing device 1 derives a required line in the fluoroscopic image according to the radius of the cup 81 and the target CE angle (S107). The radius and target CE angle of the cup 81 of the artificial hip joint 8 to be inserted into the patient K are stored in the storage unit 3 as various values ​​determined in the preoperative plan. The processing unit 2 of the information processing device 1 derives a required line in the fluoroscopic image based on a white line (acetabular sourcil) formed by the inner edge of the pelvic load surface included in the fluoroscopic image according to the radius of the cup 81 and the target CE angle. The white line (acetabular sourcil) formed by the inner edge of the pelvic load surface can be identified (the position can be grasped) by performing a shape recognition process on the fluoroscopic image. The required line indicates the minimum cutting point when cutting the pelvis of the patient K to place the cup 81.

[0044] 8 is an explanatory diagram of the required line. The storage unit 3 of the information processing device 1 stores various values ​​(parameters) determined in a preoperative plan for performing treatment related to the artificial hip joint 8, such as a minimum CE angle (e.g., 10 degrees) preset for stabilizing the cup 81 to be placed and the radius of the cup 81 to be implanted. The processing unit 2 of the information processing device 1 acquires the various values ​​determined in the preoperative plan by referring to the storage unit 3.

[0045] The processing unit 2 of the information processing device 1 defines the center (x, y) of the cup 81, which is assumed to be placed in the bone cavity (installation space) excavated by the bone excavation reamer 7, and the intersection (X, Y) of the white line (acetabular sourcil) of the inner edge of the pelvis load surface when excavated. In this case, the center (x, y) of the cup 81 is expressed by the following formula using the intersection (X, Y), the radius (r) of the cup 81, and the cup CE angle (Θ). The X coordinate (x) of the center of the cup 81 is calculated by subtracting the value obtained by multiplying the radius (r) of the cup 81 by the sine of the cup CE angle (Θ) from the X coordinate (X) of the intersection (x=Xr*sinΘ). The Y coordinate (y) of the center of the cup 81 is calculated by subtracting the value obtained by multiplying the radius (r) of the cup 81 by the cosine of the cup CE angle (Θ) from the Y coordinate (Y) of the intersection (y) (y=Yr*cosΘ). In this way, the center (x, y) of the cup 81 corresponds one-to-one to the intersection (X, Y), and when the intersection (X, Y) is changed, that is, when the white line (acetabular sourcil) due to the inner edge of the pelvis load surface at the time of drilling is changed according to the drilling amount by the bone drilling reamer 7, the center (x, y) of the cup 81 also changes. In this way, when the intersection (X, Y) is changed, the cup CE angle (Θ) must exceed the minimum CE angle (10 degrees) (Θ>10), which is a requirement for the implant, and the processing unit 2 of the information processing device 1 determines the range of change in which the intersection (X, Y) can be changed within the range that satisfies the requirement (Θ>10).

[0046] The processing unit 2 of the information processing device 1 uses the above formula to specify the area where the center (x, y) of the cup 81 can be located, based on the range of variation of the determined intersection point (X, Y). The processing unit 2 of the information processing device 1 derives the edge line of the range where (x', y') indicating each point of the outer edge of the cup 81 can be taken by taking into account the radius (r) of the cup 81 for the center (x, y) of the cup 81 determined according to the varied intersection point (X, Y), as a line (required line) that the outer edge of the bone drilling reamer 7 (the outer edge of the hemispherical tip that performs drilling) must touch in order to satisfy the requirement (Θ>10). When deriving the required line, the processing unit 2 of the information processing device 1 selects an arbitrary point (x1, y1) of the floor of the pelvis before drilling with the bone drilling reamer 7, and defines the angle of the X-axis with respect to the line connecting the center (x, y) of the cup 81 and the arbitrary point as α. The coordinates of the arbitrary point are (x+rcosα, y+rsinα). The processing unit 2 of the information processing device 1 may then derive the required line so that the point (x+rcosα, y+rsinα) taking into account the radius (r) of the cup 81 for the center (x, y) of the cup 81 corresponding to the changed intersection point (X, Y) is greater than the coordinates (x1, y1) of the arbitrary point (x+rcosα>x1 and y+rsinα>y2). That is, the processing unit 2 of the information processing device 1 derives the required line as the (x', y') range that the center (x, y) of the cup 81 that satisfies the conditional expressions "(x+rcosα>x1 and y+rsinα>y2) and "cup CE angle; Θ>10" can take, that is, the edge of the range that can be taken as the outer edge of the cup 81 by taking into account the radius (r) of the cup 81. The required reach line derived in this manner corresponds to an inner line in the reach area (the inner side on the near side in the drilling direction of the bone drilling reamer 7).

[0047] The processing unit 2 of the information processing device 1 derives an upward limit line in the fluoroscopic image according to the center of the healthy femoral head (S108). The processing unit 2 of the information processing device 1 recognizes the healthy femoral head included in the fluoroscopic image, and derives an upward limit line in the fluoroscopic image according to the center of the healthy femoral head. The upward limit line indicates a line indicating the limit of upward movement that the center of the affected femoral head (the center of the femoral head ball of the artificial hip joint 8) can take relative to the center of the healthy femoral head.

[0048] FIG. 9 is an explanatory diagram of the upper limit line. In the illustration of this embodiment, four diagrams ((1) to (4)) are shown for reference. These diagrams are as follows. Diagram (1) shows "a line parallel to the X-axis passing through y'=y2+10mm+cup radius mm (cup edge upper limit line)". Diagram (2) shows "a line parallel to the X-axis passing through y=y2+10mm (cup center upper limit line)". Diagram (3) shows "a line parallel to the X-axis passing through the healthy femoral head center (x2, y2)". Diagram (4) shows "the pelvic reference line (X-axis) connecting the lower ends of the teardrop scar". Patient K who is to undergo treatment related to an artificial hip joint 8 is generally expected to undergo treatment related to the artificial hip joint 8 (implant) on either the left or right leg.In this case, the affected side on which treatment related to the artificial hip joint 8 is to be performed is referred to as the affected side, and the healthy side on which treatment related to the artificial hip joint 8 is not to be performed is referred to as the healthy side.

[0049] The processing unit 2 of the information processing device 1 recognizes the shape of the healthy side femoral head included in the acquired fluoroscopic image (X-ray image) using an object detection model such as YOLO or edge detection, and derives the healthy side femoral head center by calculating the center of curvature, for example, from multiple points located on the outer edge based on the outer edge that forms an arc shape at the femoral head. The processing unit 2 of the information processing device 1 derives an upward limit line (line segment (2)) of the cup center (femoral head center of the affected side implant) that is within a predetermined value (for example, 10 mm) in the positive direction (upward) on the Y axis with respect to a line (line segment (3)) that passes through the derived healthy side femoral head center (x2, y2) and is parallel to the X axis (line segment (4)). The upward limit line of the cup center (femoral head center of the affected side implant) becomes a reference line for determining the cup edge upward limit line (line segment (1)), i.e., the upward limit line (line segment (1)) in the reach area (strike zone).

[0050] The center of the bone on the affected side where the treatment for the artificial hip joint 8 is performed corresponds to the center (x, y) of the cup 81 to be placed. The Y-axis indicates the upper limit direction in the human body, and the positive direction on the Y-axis indicates upward. The Y coordinate (y) of the center (x, y) of the cup 81 to be placed is set to be in the range not exceeding, for example, 10 mm upward (y < y2 + 10 [mm]) with respect to the Y coordinate (y2) of the center of the bone on the healthy side. Therefore, the processing unit 2 of the information processing device 1 passes through a point that has been moved upward (along the positive direction of the Y-axis) by the upward limit value (for example, 10 mm) with respect to the Y coordinate (y2) of the center of the bone on the healthy side, and derives an upward limit line (line segment (2)) of the cup center (the center of the bone of the implant on the affected side) that is perpendicular to the Y coordinate. Further, a cup edge upward limit line (line segment (1)), that is, an upward limit line (line segment (1)) in the reach area (strike zone), which is parallel to the upward limit line (line segment (2)) of the cup center (the center of the bone of the implant on the affected side) (that is, parallel to the X-axis) and is located upward (the positive direction on the Y-axis) by the amount of the cup radius, is determined (derived). By thus deriving the upward limit line (line segment (1): cup edge upward limit line) in the reach area (strike zone) based on the center of the bone on the healthy side, a guideline (the upper limit line in the excavation of the pelvis) can be provided to a doctor or the like when the center of the bone on the affected side (the center of the femoral head of the artificial hip joint 8) is placed.

[0051] Based on the derived inner plate line, must-reach line, and upward limit line, the processing unit 2 of the information processing device 1 derives the reach area (strike zone) (S109). Based on the derived upward limit line, inner plate line, and must-reach line, the processing unit 2 of the information processing device 1 derives the area surrounded by these lines as the reach area (strike zone).

[0052] FIG. 10 is an explanatory diagram of the reachable area (strike zone). The reachable area indicates an area where stable fixation of the cup 81 is ensured if a part of the cup 81 is in contact when the cup 81 is placed. In the reachable area, the upper limit line and the inner plate line indicate limit lines that must not be exceeded when drilling with the bone drilling reamer 7. By displaying the reachable area derived in this manner in a superimposed manner on the fluoroscopic image, the positional relationship between the current position of the bone drilling reamer 7 and the area (reachable area) that the bone drilling reamer 7 should reach by drilling can be provided to the doctor in real time by the bone drilling reamer 7 included in the fluoroscopic image displayed in real time.

[0053] The processing unit 2 of the information processing device 1 derives various actual measured values, target values, and differences between them at the current time (S110). Using the fluoroscopic image, the processing unit 2 of the information processing device 1 derives actual measured values ​​of the cup 81 that is assumed to be placed in a state in which the pelvis has been cut by the bone drilling reamer 7. The target CE angle and the target distance between the outer edge of the cup 81 and the inner plate line are set in advance as target values, and are stored in the memory unit 3 of the information processing device 1 as various values ​​determined in the preoperative plan when performing treatment related to the artificial hip joint 8. The processing unit 2 of the information processing device 1 acquires the various values ​​determined in the preoperative plan by referring to the memory unit 3.

[0054] The processing unit 2 of the information processing device 1 uses the acquired fluoroscopic image to identify the position and inclination of the cup 81 when the cup 81 is assumed to be placed in a state where the pelvis has been cut by the bone drilling reamer 7, based on the current position and inclination of the bone drilling reamer 7 (the outer edge of the hemispherical tip that performs drilling) on ​​the pelvis of the patient K. The inclination includes, for example, the anteversion angle and the abduction angle of the cup 81.

[0055] FIG. 11 is an explanatory diagram of the anteversion angle and abduction angle of a cup 81 (assumed arrangement when the pelvis is cut by the bone-drilling reamer 7). The processing unit 2 of the information processing device 1 identifies three points (A, A', B) of an ellipse by shape recognition processing. In this case, when the angle between A-A' and A'-B is X in the fluoroscopic image, the anteversion angle (anteversion angle) is expressed by "sin^(-1)*tanX". The processing unit 2 may recognize the shape and current position of the bone-drilling reamer 7 (the outer edge of the hemispherical tip that performs drilling) corresponding to the assumed arrangement of the cup 81 in the fluoroscopic image by calculation using the image coordinate system in the fluoroscopic image, and calculate the anteversion angle and abduction angle.

[0056] The processing unit 2 of the information processing device 1 may derive a target forward tilt angle and a target abduction angle based on a preset target CE angle. The processing unit 2 of the information processing device 1 may derive the target forward tilt angle and the target abduction angle by referring to a table (target CE angle table) in which the values ​​of the target forward tilt angle and the target abduction angle are associated with each value of the target CE angle. Various lookup tables, such as the target CE angle table, that the processing unit 2 of the information processing device 1 refers to when performing various calculation processes are stored in the storage unit 3.

[0057] The processing unit 2 of the information processing device 1 outputs the derived various support information (arrival area, difference between actual measurement value and target value) superimposed and attached to the perspective image (S111). The support information includes, for example, the arrival area and the difference between actual measurement value and target value. The processing unit 2 of the information processing device 1 superimposes the derived arrival area on the perspective image and outputs it to the display device 41. The processing unit 2 of the information processing device 1 further associates various actual measurement values, target values, and the difference between these actual measurement values ​​and target values ​​with the perspective image and outputs it to the display device 41. When outputting this information, the processing unit 2 of the information processing device 1 may generate and output screen data constituting a display screen (support information superimposed screen). The processing unit 2 of the information processing device 1 compares the derived actual values ​​(cup CE angle, actual distance between the outer edge of the cup 81 and the inner plate line, anteversion angle, abduction angle) with the corresponding target values ​​(target CE angle, target distance between the outer edge of the cup 81 and the inner plate line, target anteversion angle, target abduction angle), and displays them (for example, in a child screen or a separate frame) in association with the fluoroscopic image. This allows the doctor or the like to efficiently grasp the differences between various actual values ​​at the current time and the target values.

[0058] The processing unit 2 of the information processing device 1 extracts a tomographic image including the current position of the bone drilling reamer 7 from among a plurality of tomographic images of the pelvis of the patient K (S112). The processing unit 2 of the information processing device 1 performs shape recognition on the acquired fluoroscopic image using, for example, an object detection model such as YOLO or edge detection, thereby identifying the current position of the bone drilling reamer 7 (the outer edge of the hemispherical tip that performs drilling) in the pelvis of the patient K. The fluoroscopic image is aligned with a tomographic image (such as a CT image), and the processing unit 2 of the information processing device 1 can apply the internal body coordinate system defined in the tomographic image to the fluoroscopic image.

[0059] The processing unit 2 of the information processing device 1 may acquire the current position of the bone drilling reamer 7 (the outer edge of the hemispherical tip that performs drilling) included in the fluoroscopic image by specifying it in the internal body coordinate system defined by the tomographic image. The storage unit 3 of the information processing device 1 stores a plurality of tomographic images (CT images, etc.) of the patient K captured by the CT device 61 before pelvic drilling is performed with the bone drilling reamer 7. These multiple tomographic images (CT images, etc.) are images sliced ​​in the Y-axis direction, i.e., the front-back direction of the human body, and each tomographic image (CT image, etc.) is indicated by the X-axis (left-right direction of the human body) and the Z-axis (front-back direction of the human body).

[0060] The processing unit 2 of the information processing device 1 extracts one or more tomographic images (CT images, etc.) including the current position of the bone drilling reamer 7, according to the current position identified by the fluoroscopic image. The processing unit 2 of the information processing device 1 extracts a tomographic image (CT image, etc.) corresponding to the center of the bone drilling reamer 7 (the outer edge of the hemispherical tip portion that performs drilling), that is, the Y coordinate (y) of the center (x, y) of the cup 81 at the location where the cup 81 is assumed to be placed in the current drilling state. Alternatively, since each of the tomographic images is associated with the coordinate of the Y axis and the area (range) of the XZ plane corresponding to the tomographic image, for example, using an internal body coordinate system, the processing unit 2 of the information processing device 1 may extract a tomographic image corresponding to the current position of the bone drilling reamer 7 in the internal body coordinate system.

[0061] The processing unit 2 of the information processing device 1 superimposes a cup periphery diagram when the position of the cup 81 is assumed when the pelvis is cut by the bone drilling reamer 7 on the extracted tomographic image and outputs it (S113). The processing unit 2 of the information processing device 1 superimposes a semicircular figure (cup periphery diagram) indicating the current position of the bone drilling reamer 7 (the outer edge of the hemispherical tip part that performs drilling) on ​​one or more extracted tomographic images (CT images, etc.) and outputs it. The processing unit 2 of the information processing device 1 may further use the perspective image to derive the installation angle and depth of the cup 81 at the place where the cup 81 is assumed to be installed in the current drilling state, and further superimpose it on the tomographic image (CT image, etc.) and output it. In this way, the processing unit 2 of the information processing device 1 extracts a corresponding tomographic image (CT image, etc.) according to the current position of the bone drilling reamer 7 specified in the fluoroscopic image (the place where the cup 81 is expected to be placed in the current drilling state), and outputs the tomographic image by superimposing an outer edge diagram (semicircular graphic object) of the cup 81 expected to be placed, and the installation angle and depth of the cup 81. This makes it possible to provide a doctor or the like with information to support three-dimensional intraoperative recognition, including information about the Z axis (the front-back direction in the human body).

[0062] 12 is an explanatory diagram illustrating a display screen of support information (support information superimposed screen). As a result of the various processes described above, the processing unit 2 of the information processing device 1 generates screen data constituting the support information superimposed screen, and outputs the screen data to the display device 41. The support information superimposed screen includes a perspective image display area, a tomographic image display area, and a support information display area.

[0063] The fluoroscopic image display area displays a fluoroscopic image (X-ray image) of the pelvis of patient K captured in real time during surgery while the pelvis is being cut with the bone drilling reamer 7, and the reachable area is superimposed on the fluoroscopic image. The tomographic image display area displays a tomographic image (CT image) including the current position of the bone drilling reamer 7 identified in the fluoroscopic image, and the outer edge diagram (semicircular graphic object) of the cup 81 when the cup 81 is assumed to be positioned in a state where the pelvis has been cut with the bone drilling reamer 7 is superimposed on the tomographic image. The reachable area may also be superimposed on the tomographic image.

[0064] The support information display area displays in list form the anteversion angle, abduction angle, cup CE angle, and remaining distance from the inner plate line of the assumedly positioned cup 81 when the pelvis has been scraped by the bone drilling reamer 7. For the anteversion angle, abduction angle, cup CE angle, and remaining distance from the inner plate line of the cup 81, actual measurement values ​​calculated based on the current position and inclination of the bone drilling reamer 7 identified by shape recognition processing in the fluoroscopic image, values ​​(target values) determined in the preoperative plan, and the difference between the actual measurement values ​​and the target values ​​are displayed in list form.

[0065] 13 is an explanatory diagram illustrating a display screen (distortion correction screen) of support information. As a result of the various processes described above, the processing unit 2 of the information processing device 1 generates screen data constituting the distortion correction screen, and outputs the screen data to the display device 41. The distortion correction screen includes a display area before distortion correction and a display area after distortion correction. In this way, the distortion correction screen displays a perspective image before distortion correction (with support information superimposed) and a perspective image after distortion correction (with support information superimposed) in comparison, so that it is possible to present information indicating the state before and after distortion correction for the reachable area (strike zone) to a doctor or the like, and to provide the doctor with information useful for performing treatment.

[0066] The distortion correction screen may be output as a separate screen from the support information superimposed screen described above, or may be included in the support information superimposed screen. When the distortion correction screen is output as a separate screen from the support information superimposed screen, the display device 41 displaying the distortion correction screen and the display device 41 displaying the support information superimposed screen may be separate display devices 41. In this case, two display devices 41, one displaying the distortion correction screen and the other display device 41 displaying the support information superimposed screen, are connected to the information processing device 1.

[0067] The image displayed in the perspective image display area on the support information superimposed screen (the perspective image with the reachable area superimposed thereon) is displayed in the pre-distortion correction display area. Furthermore, the image displayed in the pre-distortion correction display area (the pre-correction image) displays a lattice-like auxiliary line made up of a plurality of curved lines used for distortion correction.

[0068] The image after distortion correction (the perspective image with the reach area superimposed) is displayed in the display area after distortion correction. Furthermore, the image (the corrected image) displayed in the display area after distortion correction displays the auxiliary lines used in the distortion correction process in a state in which they have been transformed into straight lines in accordance with the correction. As a result, the reach area (strike zone) is also displayed in a state in which its shape (the area in the pelvis) has been changed in accordance with the distortion correction.

[0069] When performing distortion correction, the processing unit 2 of the information processing device 1 may, for example, perform distortion correction processing by attaching a template including a straight line under a perspective image (image before correction). In this case, the processing unit 2 of the information processing device 1 is a device that converts X-rays that have passed through the human body into a digital image, and may use a flat panel having a distortion correction function.

[0070] Alternatively, the processing unit 2 of the information processing device 1 may perform a distortion correction process on a fluoroscopic image (image before correction) according to a distortion aberration determined based on an X-ray imaging characteristic of an X-ray device 62 or the like. The X-ray device 62 that captures a fluoroscopic image (X-ray image) has various characteristics according to the model or type, and can also specify a distortion aberration corresponding to the amount of distortion in the captured fluoroscopic image (X-ray image). The storage unit 3 of the information processing device 1 stores parameters such as distortion aberration corresponding to each of various X-ray devices 62, and the processing unit 2 of the information processing device 1 may perform distortion correction using parameters such as distortion aberration corresponding to the X-ray device 62 or the like. In this case, it is not necessary to use a flat panel that is a dedicated device for performing distortion correction by hardware processing, and a relatively inexpensive display (a display device without a distortion correction function by hardware processing) can be used to display the fluoroscopic image after distortion correction.

[0071] According to this embodiment, the processing unit 2 of the information processing device 1 acquires a fluoroscopic image (X-ray image) of the patient K undergoing a treatment related to the artificial hip joint 8. The fluoroscopic image is, for example, an X-ray image. When a bone-drilling reamer 7 is used to form a bone recess (installation space) that serves as a pelvis receptacle for installing the cup 81 of the artificial hip joint 8 (drilling the pelvis) as a treatment related to the artificial hip joint 8, the fluoroscopic image (X-ray image) is captured as a moving image in real time, and the processing unit 2 of the information processing device 1 sequentially acquires the fluoroscopic images (X-ray images) captured in real time. The processing unit 2 of the information processing device 1 recognizes (derives) an inner plate line indicating the maximum cutting point when cutting the pelvis to place the cup 81 including the artificial hip joint 8, and a must-reach line (a line to be exceeded) indicating the minimum cutting point when cutting the pelvis to place the cup 81, in the pelvis of the patient K in the acquired fluoroscopic image. Furthermore, the processing unit 2 of the information processing device 1 derives a cup CE angle when it is assumed that the cup 81 will be placed in a state where the pelvis has been scraped by the bone drilling reamer 7 (in real time). The processing unit 2 of the information processing device 1 superimposes the recognized (derived) inner plate line and the required line on the fluoroscopic image, and displays the derived cup CE angle on, for example, a child screen, etc., in association with the fluoroscopic image, so that it is possible to provide useful support information to a doctor who performs a procedure on the artificial hip joint 8 for the patient K. The processing unit 2 of the information processing device 1 recognizes the shape of the outer edge (the outer edge of the hemispherical tip part that performs the drilling) of the bone drilling reamer 7 included in the fluoroscopic image (X-ray image) using an object detection model such as YOLO or edge detection. The processing unit 2 of the information processing device 1 regards the outer edge of the bone drilling reamer 7 as the outer edge of the cup 81, and derives a cup CE angle when it is assumed that the cup 81 will be placed in a state where the pelvis has been scraped by the bone drilling reamer 7. The processing unit 2 of the information processing device 1 derives the cup CE angle as the angle formed by a line connecting the center of rotation (COR) of the cup 81, which is assumed to be positioned when the pelvis has been cut by the bone-drilling reamer 7, to the intersection of the outer edge of the cup 81 and the white line (acetabular sourcil) of the pelvic load surface, and a line connecting the lower ends of the left and right teardrops on the pelvis, which is perpendicular to the pelvic reference line (Y-axis).The processing unit 2 of the information processing device 1 recognizes the shape of the lower ends of the left and right teardrops included in the fluoroscopic image (X-ray image) using, for example, an object detection model such as YOLO or edge detection processing, and identifies a pelvis reference line passing through the lower ends of the left and right teardrops. The processing unit 2 of the information processing device 1 identifies a vertical line that is perpendicular to the identified pelvis reference line and passes through the center (COR) of the cup 81 that is assumed to be placed in a state in which the pelvis has been scraped by the bone drilling reamer 7. The processing unit 2 of the information processing device 1 derives the cup CE angle by calculating the angle of a line connecting the center of the cup 81 to the intersection of the outer edge of the cup 81 and the white line (acetabular sourcil) of the pelvis load surface with respect to the vertical line passing through the center (COR) of the cup 81. The processing unit 2 of the information processing device 1 acquires a fluoroscopic image captured in real time, and derives the cup CE angle based on the outer edge of the bone drilling reamer 7 included in the acquired fluoroscopic image. This makes it possible to track the current position of the bone-drilling reamer 7, i.e., the drilling state by the bone-drilling reamer 7, and provide the surgeon with the cup CE angle when the cup 81 is currently placed at any time. In other words, it is possible to configure an intraoperative support system S that enables real-time intraoperative evaluation of the placement of the cup 81 and the coverage rate.

[0072] According to this embodiment, one or more tomographic images of the pelvis of the patient K are, for example, CT images or MRI images, and are taken before the pelvis of the patient K is drilled by the bone drilling reamer 7. The processing unit 2 of the information processing device 1 acquires these multiple tomographic images and stores them in the storage unit 3. The storage unit 3 of the information processing device 1 stores an entity file of a learning model 101 (inner plate line model) that has learned to output position information (landmarks) of the inner plate when a tomographic image is input. The processing unit 2 of the information processing device 1 inputs the acquired tomographic image (CT image, etc.) to the learning model 101 (inner plate line model), and the learning model 101 (inner plate line model) outputs the position information of the inner plate by superimposing it on the tomographic image. Each of the position information of the inner plate is output as a point (landmark), and a plurality of points (landmarks) are output on the tomographic image, and a line connecting these multiple points (landmarks) is formed (connected) to specify the inner plate line. In this way, when a tomographic image is input, the learning model 101 (inner plate line model) that has been trained to output position information (landmarks) of the inner plate can be used to efficiently identify (derive) the inner plate line that indicates the maximum cutting point when cutting the pelvis to place the cup 81 containing the artificial hip joint 8. The processing unit 2 of the information processing device 1 aligns the tomographic image (CT image, etc.) and the fluoroscopic image (X-ray image) based on the same internal body part (e.g. teardrop) identified using shape recognition or the like, and can define these two images (tomographic image and fluoroscopic image) in the same internal body coordinate system and perform various calculation processes such as superimposition display on the images. The processing unit 2 of the information processing device 1 displays (outputs) the inner plate line derived using the tomographic image (CT image, etc.) superimposed on the fluoroscopic image (X-ray image) so that it is at the same position (internal body coordinate value) as the inner plate line shown in the tomographic image. The fluoroscopic image (X-ray image) displayed in real time during surgery shows the bone drilling reamer 7 at the current time, and the inner plate line is superimposed on it, so that the doctor or other person operating the bone drilling reamer 7 can effectively be shown the inner plate line (drilling limit line) that is connected at the maximum cutting point when cutting the pelvis, i.e., the limit point that must not be exceeded.

[0073] The embodiments disclosed herein are illustrative in all respects and should not be considered as limiting. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the scope and meaning equivalent to the claims.

[0074] The claims may be combined with each other regardless of the form of reference. The claims may contain multiple dependent claims depending on multiple claims. Multiple dependent claims may be contained depending on multiple dependent claims. If multiple dependent claims are not contained depending on a multiple dependent claim, this does not limit the number of dependent claims depending on a multiple dependent claim. [Explanation of symbols]

[0075] S Intraoperative Support System K patient 1. Information processing device 2 Processing section 3 Storage section M Recording medium P Program (Program Product) 4 Input / Output Interface 41 Display device 5. Communications Department 101 Learning model (inner board line model) 61 CT scanner (tomographic imaging device) 62 X-ray equipment (fluoroscopic imaging equipment) 7 Bone Drilling Reamers 8. Artificial Hip Joint 81 Cups

Claims

1. On the computer, Obtaining fluoroscopic images of a patient undergoing a procedure relating to an artificial hip joint; Recognizing an inner plate line indicating the maximum cutting point when cutting the pelvis in order to place a cup included in the artificial hip joint in the patient's pelvis in the acquired fluoroscopic image, Recognizing a line of minimum distance to be traveled in the patient's pelvis to place the cup; A cup CE angle is derived when the patient's pelvis is scraped with a bone drilling reamer used in the treatment related to the artificial hip joint, assuming the placement of the cup; The recognized inner plate line and the required line are superimposed on the perspective image, The derived cup CE angle is displayed in association with the fluoroscopic image. A program that executes a process.

2. Acquire a tomographic image of the patient's pelvis; A learning model that has been trained to output position information of an inner plate when a tomographic image is inputted, deriving position information of the inner plate by inputting the acquired tomographic image, The inner plate line in the perspective image is recognized based on the output group of position information of the plurality of inner plates. The program according to claim 1.

3. Based on the femoral head center of the healthy side not treated with the artificial hip joint, an upper limit line is derived for the femoral head center of the affected side to be treated with the artificial hip joint; Derive a reachable region based on the derived upward limit line, the inner plate line, and the required reachable line; The derived reachable area is superimposed on the perspective image. The program according to claim 1.

4. The required line is derived based on a target CE angle that is determined in advance to stabilize the cup to be placed, an intersection point between an outer edge of the cup and an inner edge of a load surface in the patient's pelvis that serves as a reference when ensuring the target CE angle, and a radius of the cup. The program according to claim 1.

5. The derived cup CE angle is compared with a target CE angle that is preset as a target value, and the result is output. Derive an actual distance between an outer edge of the cup and the inner plate line when assuming the placement of the cup in a state in which the patient's pelvis is cut by the bone drilling reamer; Derive a target distance between the outer edge of the cup and the inner plate line, the target distance being preset as a target value; The derived actual distance is compared with the target distance and output. The program according to claim 1.

6. Deriving an anteversion angle and an abduction angle of the cup when assuming the position of the cup in a state in which the patient's pelvis is cut by the bone drilling reamer; Output the derived forward tilt angle and abduction angle The program according to claim 1.

7. Deriving a target forward tilt angle and a target abduction angle according to the target CE angle; The derived target forward tilt angle and the derived target abduction angle are combined and output. The program according to claim 5.

8. The derived anteversion angle and abduction angle of the cup are compared with the target anteversion angle and the target abduction angle and output. The program according to claim 7.

9. obtaining a current position of the bone reamer in the patient's pelvis based on the obtained fluoroscopic image; extracting a tomographic image including a current position of the bone drilling reamer from among the plurality of tomographic images of the patient's pelvis; A diagram of the outer edge of the cup when the arrangement of the cup is assumed in a state where the patient's pelvis has been cut by the bone drilling reamer is superimposed on the extracted tomographic image and output. The program according to claim 2.

10. On the computer, Obtaining fluoroscopic images of a patient undergoing a procedure relating to an artificial hip joint; Recognizing an inner plate line indicating the maximum cutting point when cutting the pelvis in order to place a cup included in the artificial hip joint in the patient's pelvis in the acquired fluoroscopic image, Recognizing a line of minimum distance to be traveled in the patient's pelvis to place the cup; A cup CE angle is derived when the patient's pelvis is scraped with a bone drilling reamer used in the treatment related to the artificial hip joint, assuming the placement of the cup; The recognized inner plate line and the required line are superimposed on the perspective image, The derived cup CE angle is displayed in association with the fluoroscopic image. An information processing method for executing a process.

11. An acquisition unit for acquiring fluoroscopic images of a patient undergoing a procedure related to an artificial hip joint; An inner plate line recognition unit that recognizes an inner plate line indicating a maximum cutting point when cutting the pelvis in order to place a cup included in the artificial hip joint in the patient's pelvis in the acquired fluoroscopic image; a required line recognition unit that recognizes a required line indicating a minimum cutting point when cutting the pelvis of the patient in order to place the cup; A deriving unit for deriving a cup CE angle when assuming the placement of the cup in a state where the patient's pelvis is cut by a bone drilling reamer used in a treatment related to the artificial hip joint; a display unit that displays the recognized inner plate line and the required line superimposed on the fluoroscopic image and displays the derived cup CE angle in association with the fluoroscopic image; An information processing device comprising:

Images