Program, information processing method, and information processing apparatus

Abstract

An inner plate line indicating a maximum shaving point when shaving a pelvis to arrange a cup included in an artificial hip joint is displayed in a superimposed manner.SOLUTION: Acquiring, by a computer, a fluoroscopic image of a patient to be subjected to a procedure related to an artificial hip joint, recognizing, in a pelvis of the patient in the acquired fluoroscopic image, an inner plate line indicating a maximum cutting point when cutting the pelvis to arrange a cup included in the artificial hip joint, and recognizing, in the pelvis of the patient, an indispensable line indicating a minimum cutting point when cutting the pelvis to arrange the cup; And deriving a cup CE angle when the cup is assumed to be placed in a state in which the pelvis of the patient has been shaved by the bone drilling reamer used in the treatment related to the artificial hip joint, superimposing and displaying the recognized medial plate line and the requisite line on the fluoroscopic image, and displaying the derived cup CE angle in association with the fluoroscopic image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a program, an information processing method, and an information processing device. [Background technology]

[0002] An acetabular socket angle setting device for use in total hip replacement surgery is known (for example, Patent Document 1). The acetabular socket angle setting device for use in total hip replacement surgery described in Patent Document 1 provides a method for setting the acetabular socket angle based on the pelvic surface and the inter-tear line. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-031047 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the acetabular socket angle setting device for total hip replacement surgery in Document 1 does not take into consideration the need to superimpose an inner plate line, which indicates the maximum cutting point when cutting the pelvis to place the cup included in the artificial hip joint, on a fluoroscopic image of a patient undergoing treatment related to the artificial hip joint.

[0005] The present disclosure has been made in consideration of the above circumstances, and aims to provide a program or the like that can superimpose an inner plate line, which indicates the maximum cutting point when cutting the pelvis to place the cup included in the artificial hip joint, on an X-ray image of a patient undergoing treatment related to an artificial hip joint. [Means for solving the problem]

[0006] In one proposal, the program causes a computer to execute the following processes: acquire an fluoroscopic image of a patient undergoing a procedure related to an artificial hip joint; recognize an inner plate line in the patient's pelvis in the acquired fluoroscopic image, which indicates the point at which the pelvis needs to be removed to position a cup included in the artificial hip joint; recognize a required line in the patient's pelvis, which indicates the point at which the pelvis needs to be removed to position the cup; derive a cup CE angle when it is assumed that the cup will be positioned in a state in which the patient's pelvis has been removed using a bone drilling reamer used in the procedure related to the artificial hip joint; superimpose the recognized inner plate line and required line on the fluoroscopic image; and display the derived cup CE angle in association with the fluoroscopic image.

[0007] In one proposal, the information processing method causes a computer to execute a process of acquiring an fluoroscopic image of a patient undergoing a procedure related to an artificial hip joint, recognizing an inner plate line in the patient's pelvis in the acquired fluoroscopic image, which indicates the point at which the pelvis needs to be removed to position a cup included in the artificial hip joint, recognizing a required line in the patient's pelvis, which indicates the point at which the pelvis needs to be removed to position the cup, deriving a cup CE angle when it is assumed that the cup will be positioned in a state in which the patient's pelvis has been removed using a bone drilling reamer used in the procedure related to the artificial hip joint, superimposing the recognized inner plate line and required line on the fluoroscopic image, and displaying the derived cup CE angle in association with the fluoroscopic image.

[0008] In one proposal, the information processing device comprises an acquisition unit that acquires an fluoroscopic image of a patient undergoing a procedure related to an artificial hip joint; an inner plate line recognition unit that recognizes an inner plate line in the patient's pelvis in the acquired fluoroscopic image, which indicates the point at which the pelvis needs to be removed to place the cup included in the artificial hip joint; a required line recognition unit that recognizes a required line in the patient's pelvis, which indicates the point at which the pelvis needs to be removed to place the cup; a derivation unit that derives the cup CE angle when it is assumed that the patient's pelvis will be removed using a bone drilling reamer used in the procedure related to the artificial hip joint, and a display unit that superimposes the recognized inner plate line and required line on the fluoroscopic image and displays the derived cup CE angle in association with the fluoroscopic image. [Effects of the Invention]

[0009] According to the present disclosure, it is possible to superimpose an inner plate line indicating the maximum point of reduction when reducing the pelvis to place the cup included in the artificial hip joint on a fluoroscopic image of a patient undergoing treatment related to an artificial hip joint. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a schematic diagram showing an overview of an intraoperative support system including an information processing device according to a first embodiment. [Figure 2] FIG. 1 is a block diagram illustrating an example of the configuration of an information processing device. [Figure 3] FIG. 10 is an explanatory diagram showing an example of a learning model (inner plate line model). [Figure 4] 10 is a flowchart showing an example of a processing procedure (during model learning) of a processing unit of the information processing device. [Figure 5] 10 is a flowchart showing an example of a processing procedure (during model operation) of a processing unit of the information processing device. [Figure 6] FIG. 1 is an explanatory diagram regarding the cup CE angle. [Figure 7] FIG. 10 is an explanatory diagram of an inner plate line. [Figure 8] FIG. 10 is an explanatory diagram of a required line. [Figure 9] FIG. 10 is an explanatory diagram regarding an upward limit line. [Figure 10] FIG. 1 is an explanatory diagram of a strike zone. [Figure 11] FIG. 10 is an explanatory diagram of the anteversion angle and abduction angle of the cup (assumed position after the pelvis has been removed with a bone-drilling reamer). [Figure 12] FIG. 10 is an explanatory diagram illustrating an example of a display screen of support information (a support information superimposed screen). [Figure 13] FIG. 10 is an explanatory diagram illustrating an example of a display screen (distortion correction screen) for support information. DETAILED DESCRIPTION OF THE INVENTION

[0011] The present invention will be described in detail below with reference to the drawings showing embodiments thereof. (Embodiment 1)

[0012] FIG. 1 is a schematic diagram showing an overview of an intraoperative support system S including an information processing device 1 according to the first embodiment. FIG. 2 is a block diagram showing an example of the configuration of the information processing device 1. The intraoperative support system S is configured with the information processing device 1 as a main device, and a fluoroscopic image capturing device such as an X-ray device 62 and a tomographic image capturing device such as a CT device 61 are communicably connected to the information processing device 1. The information processing device 1 acquires X-ray images (fluoroscopic images) captured by the X-ray device 62 in real time. The information processing device 1 acquires CT images (tomographic images) captured by the CT device 61. The information processing device 1 may further be communicably connected to an electronic medical record server that stores and manages various medical data related to patient K.

[0013] A CT image (cross-sectional image) of patient K, who is to undergo treatment for an artificial hip joint 8, is taken by the CT device 61 before (preoperatively) the pelvis is drilled with the bone drilling reamer 7. An X-ray image (fluoroscopic image) of patient K, who is to undergo treatment for an artificial hip joint 8, is taken by the X-ray device 62 while (intraoperatively) the pelvis is drilled with the bone drilling reamer 7. Therefore, the X-ray image includes the bone drilling reamer 7 drilling the pelvis (during drilling).

[0014] After a doctor has properly drilled the pelvis using a bone drilling reamer 7, an artificial hip joint 8 including a cup 81 is inserted (implanted) into the body of patient K. The artificial hip joint 8 includes a hemispherical cup 81 that is fitted to the drilled pelvis, and a stem that is inserted into the femur. The bone drilling reamer 7 has a hemispherical tip that is used for drilling, and the outer edge shape of the tip is the same shape and size as the outer edge shape of the cup 81 of the artificial hip joint 8.

[0015] Although details will be described later, the processing unit 2 of the information processing device 1 recognizes the shape of the bone drilling reamer 7 (the hemispherical tip that drills) included in the acquired X-ray image using, for example, an object detection model such as YOLO or edge detection, and identifies the position and inclination of the bone drilling reamer 7 in the pelvis. The processing unit 2 of the information processing device 1 regards the identified position and inclination of the bone drilling reamer 7 as the position and inclination of the cup 81 that is expected to be placed when the pelvis has been cut by the bone drilling reamer 7, and calculates and outputs various information related to the cup 81 that is expected to be placed. Furthermore, the processing unit 2 of the information processing device 1 calculates and outputs various information related to the cup 81 that is expected to be placed by using a learning model 101 (inner plate line model) for multiple CT images taken before surgery. Using the landmarks, an inner plate line (reachable area) for assisting drilling by the bone drilling reamer 7 is superimposed on the X-ray image.

[0016] The information processing device 1 is a computer capable of various information processing and sending and receiving information, such as a server device or a personal computer. The server device includes not only a single server device but also a cloud server device or a virtual server device configured by multiple computers. When the information processing device 1 is configured as, for example, a cloud server device, the information processing device 1 does not need to be installed in a medical facility where the patient K is located, like medical devices such as a CT device 61 or an X-ray device 62, but may be connected to these medical devices via an external network such as the Internet so as to be able to communicate with them. The information processing device 1 includes a processing unit 2, a storage unit 3, an input / output I / F 4, and a communication unit 5.

[0017] The processing unit 2 has an arithmetic processing device equipped with a timing function such as one or more CPUs (Central Processing Units), MPUs (Micro-Processing Units), GPUs (Graphics Processing Units), etc., and performs various information processing, control processing, etc. related to the information processing device 1 by reading and executing a program P (program product) stored in the memory unit 3.

[0018] The storage unit 3 includes a volatile storage area such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), or flash memory, and a non-volatile storage area such as EEPROM or a hard disk. The storage unit 3 pre-stores a program P (program product) and data to be referenced during processing. The program P (program product) stored in the storage unit 3 may be a program P (program product) read from a recording medium M readable by the information processing device 1. Alternatively, the program P (program product) may be downloaded from an external computer (not shown) connected to a communication network (not shown) and stored in the storage unit 3.

[0019] The memory unit 3 stores various medical data related to patient K and various values ​​determined in the preoperative plan. The memory unit 3 also stores an entity file that constitutes a learning model 101 (interior plate line model). The entity file may be configured as a part of a program P (program product).

[0020] The communication unit 5 is a communication module or communication interface for communicating, via wired or wireless means, with an electronic medical record server or an information terminal such as a smartphone held by a medical professional, and is, for example, a wired communication module such as an Ethernet (registered trademark) connector, a short-range wireless communication module such as WiFi (registered trademark) or Bluetooth (registered trademark), or a wide-area wireless communication module such as 4G or 5G. The processing unit 2 communicates with the electronic medical record server or the information terminal via the communication unit 5, for example, via a local network within the medical institution or an external network such as the Internet.

[0021] The input / output I / F 4 is a communication interface that complies with a communication standard such as RS232C or USB. An input device such as a keyboard or a display device 41 such as a liquid crystal display is connected to the input / output I / F 4. Furthermore, medical equipment such as a CT device 61 or an X-ray device 62 may be connected to the input / output I / F 4.

[0022] 3 is an explanatory diagram showing an example of a learning model 101 (inner plate line model). The learning model 101 (inner plate line model) is a neural network (NN) that is configured using, for example, RCNN (Regions with Convolutional Neural Network), Fast RCNN, Faster RCNN, SSD (Single Shot Multibook Detector), YOLO (You Only Look Once), or the like, and performs object detection, semantic segmentation, or instance segmentation.

[0023] The learning model 101 (inner plate line model) determines whether or not an inner plate corresponding point is included (presence / absence) based on an input image (a tomographic image such as a CT image), and if an inner plate corresponding point is included (presence), outputs the region (position information) of the inner plate corresponding point in the input image. In other words, the learning model 101 functions as a region extraction model that extracts the region of the inner plate corresponding point included in the input image (a tomographic image such as a CT image).

[0024] When the learning model 101 (inner plate line model) is configured with a neural network including a CNN (Convolutional Neural Network) that extracts image features, such as RCNN, the input layer included in the learning model 101 (inner plate line model) has multiple neurons that accept input of pixel values ​​of the image and passes the input pixel values ​​to the intermediate layer. The intermediate layer has multiple neurons that extract image features of the image and passes the extracted image features to the output layer. The output layer has one or multiple neurons that output region information including the positions of inner plate corresponding points, and outputs the positions of the inner plate corresponding points (region coordinates or pixel numbers, etc.) based on the image features output from the intermediate layer.

[0025] The neural network (learning model 101) trained using training data is expected to be used as a program module that is part of artificial intelligence software. The learning model 101 is used in the information processing device 1 that includes the processing unit 2 (CPU, etc.) and the storage unit 3 as described above, and when executed by the information processing device 1 having such computational processing capabilities, a neural network system is configured. That is, the processing unit 2 of the information processing device 1 performs computation to extract feature quantities of the image input to the input layer in accordance with instructions from the learning model 101 stored in the storage unit 3, and outputs the positions (areas) of the inner plate corresponding points from the output layer.

[0026] The learning model 101 (internal plate line model) can be generated by preparing training data in which tomographic images (problem data), such as CT images including the pelvis (hip joint), are associated with labels (answer data) indicating the positions (areas) of internal plate corresponding points, which are points on the internal plate line in the pelvis, and then using the training data to train an untrained neural network. The training data is stored, for example, in the storage unit 3 of the information processing device 1 and can be generated by aggregating images and doctor's findings contained in a large number of diagnostic or surgical results stored on an electronic medical record server or the like at a medical institution such as a hospital. In other words, the internal plate line in the pelvis (internal plate corresponding points, which are points on the internal plate line) is a location (internal body part) identified based on the findings of a doctor or the like. According to the learning model 101 (internal plate line model) trained and configured in this way, by inputting a tomographic image, such as a CT image, into the learning model 101, information indicating the positions (area coordinates in the image coordinate system) of the internal plate corresponding points contained in the tomographic image can be obtained.

[0027] In this embodiment, the learning model 101 (inner plate line model) has been described as being RCNN, but the learning model 101 is not limited to RCNN and may be a learning model 101 constructed with other learning algorithms, such as a neural network other than RCNN, an SVM (Support Vector Machine), a Transformer, YOLO, a Bayesian network, or a regression tree. The dataset of question data and answer data included in the training data for learning the learning model 101 and the dataset of input data and output data when using the learning model 101 are synonymous, and if defined in one dataset, it naturally applies to the other dataset as well.

[0028] 4 is a flowchart showing an example of a processing procedure (during model learning) of the processing unit 2 of the information processing device 1. The processing unit 2 of the information processing device 1 receives an operation by an operator using, for example, a keyboard connected to the input / output I / F 4, and performs the following processing based on the received operation.

[0029] The processing unit 2 of the information processing device 1 acquires training data (S11). The processing unit 2 of the information processing device 1 acquires training data to which position information of inner plate corresponding points (landmarks) indicating points on the inner plate line is added for a tomographic image (e.g., a CT image). The position information of the inner plate corresponding points (landmarks) may be, for example, information obtained by annotating a shape such as a circle at a location (pelvic region) indicating the inner plate corresponding point for the tomographic image.

[0030] The processing unit 2 of the information processing device 1 uses the acquired training data to generate a learning model 101 (inner plate line model) (S12). The processing unit 2 of the information processing device 1 inputs training data (images in which the tomographic images are annotated at locations indicating inner plate corresponding points) to an untrained neural network, thereby training the neural network and generating the learning model 101 (inner plate line model).

[0031] In this embodiment, the learning model 101 outputs position information of inner plate corresponding points (landmarks) indicating points on the inner plate line based on the input CT image (superimposing the inner plate corresponding points (landmarks) on the CT image), but this is not limited to this. The learning model 101 may also output, based on the input CT image, required-attainment corresponding points indicating points on the required-attainment line (described later) and upward limit points indicating points on the upward limit line. In this case, a target CE angle may be input to the learning model 101 in addition to the CT image. That is, the learning model 101 may function as a reachable area model that outputs points on the inner plate line, required-attainment line, and upward limit line (inner plate corresponding points, required-attainment corresponding points, and upward limit points) for identifying the reachable area based on the input CT image (or the CT image and the target CE angle). In this case, the reachable region model is trained using training data including answer data annotated with inner plate corresponding points, required corresponding points, and upper limit points for the CT image serving as problem data.

[0032] 5 is a flowchart showing an example of a processing procedure (during model operation) of the processing unit 2 of the information processing device 1. The processing unit 2 of the information processing device 1 starts the processing of this flowchart, for example, when triggered by input of a fluoroscopic image or when receiving a start command or the like from an input device such as a keyboard connected to the input / output I / F 4.

[0033] The processing unit 2 of the information processing device 1 acquires a tomographic image (CT image) of the pelvis of patient K (S101). Prior to (preoperatively) a drilling procedure using the bone drilling reamer 7, the patient K has had tomographic images (CT images or MRI images) of internal body parts including the pelvis and hip joints captured using, for example, a CT device 61 or an MRI device. The tomographic images captured preoperatively are stored, for example, in the storage unit 3 of the information processing device 1. Alternatively, these tomographic images (e.g., CT images) may be stored in an electronic medical record server, and the processing unit 2 of the information processing device 1 may access the electronic medical record server using, for example, a patient KID that uniquely identifies patient K, and acquire the tomographic images (e.g., CT images) from the electronic medical record server. The tomographic image includes multiple tomographic images sliced ​​at a predetermined slice width (in the vertical (Y-axis) direction of the human body). That is, in this embodiment, the Y-axis indicates the upper limit direction of the human body, and the positive direction of the Y-axis indicates upward. The X-axis indicates the left-right direction of the human body (positive direction is to the right on the paper), and the Z-axis indicates the front-to-back direction of the human body (positive direction is forward).

[0034] The processing unit 2 of the information processing device 1 inputs the tomographic image to the learning model 101 (inner plate line model) (S102). The processing unit 2 of the information processing device 1 inputs each of the acquired multiple tomographic images to the learning model 101 (inner plate line model). The learning model 101 (inner plate line model) has been trained to output position information (landmarks) of the inner plate (inner plate corresponding points on the inner plate line) when a tomographic image is input. The learning model 101 (inner plate line model) outputs the position information (landmarks) of the inner plate by superimposing them on the input tomographic image based on the input tomographic image.

[0035] The processing unit 2 of the information processing device 1 acquires inner plate corresponding points (landmarks) indicating points on the inner plate line from the learning model 101 (S103). The processing unit 2 of the information processing device 1 acquires, from the learning model 101, tomographic images on which the inner plate corresponding points (landmarks) indicating points on the inner plate line are superimposed, and stores the acquired multiple tomographic images (tomographic images on which the inner plate corresponding points are superimposed) in the memory unit 3.

[0036] The processing unit 2 of the information processing device 1 acquires various values ​​determined in the preoperative plan by referring to the storage unit 3 (S104). The various values ​​determined in the preoperative plan include, for example, the radius of the cup 81 of the artificial hip joint 8 to be placed inside the body, the target CE angle, the target distance between the outer edge of the cup 81 and the medial plate line, the target anteversion angle, and the target abduction angle, which are set in advance as target values.

[0037] The target CE angle indicates the cup CE angle when placing the cup 81, which is set in advance as a target value. The target distance indicates the distance between the cup 81, which is set in advance as a target value, and the inner plate line derived using the learning model 101 (inner plate line model) based on the tomographic image. The target anterior tilt angle indicates the anteversion angle of the cup 81 when placing the cup 81, which is set in advance as a target value. The target abduction angle indicates the abduction angle of the cup 81 when placing the cup 81, which is set in advance as a target value. These various values ​​determined in the preoperative plan are determined by a doctor or the like, input into the information processing device 1 in advance, and stored in the memory unit 3. Alternatively, the processing unit 2 of the information processing device 1 may acquire the various values ​​determined in the preoperative plan for the patient K from an electronic medical record system based on the patient KID or the like.

[0038] The processing unit 2 of the information processing device 1 acquires a fluoroscopic image (X-ray image) of the pelvis of patient K during surgery for the patient K undergoing a procedure related to the artificial hip joint 8 (S105). The processing unit 2 of the information processing device 1 acquires a fluoroscopic image (X-ray image) of the pelvis of the patient K in real time during surgery for the patient K undergoing a procedure related to the artificial hip joint 8, i.e., while the pelvis is being drilled with the bone drilling reamer 7. The processing unit 2 of the information processing device 1 acquires the fluoroscopic image (X-ray image) in real time, and may acquire it in the form of a moving image. In this way, the fluoroscopic image (X-ray image) of the patient K acquired in real time during surgery includes the bone drilling reamer 7 drilling the pelvis, and the current position and inclination of the bone drilling reamer 7 (the outer edge of the hemispherical tip that drills) in the pelvis can be identified. When the placement of the cup 81 is assumed to be in the current position of the bone-drilling reamer 7 in the pelvis, i.e., in the bone depression (installation space) excavated by the bone-drilling reamer 7, the cup CE angle, anteversion angle, and abduction angle of the assumedly placed cup 81 can be derived.

[0039] 6 is an explanatory diagram of the cup CE angle. In the illustration of this embodiment, a semicircular graphic object representing a cup 81 assumed to be placed based on the current position of the bone excavation reamer 7 in the pelvis is displayed superimposed on a perspective image. The processing unit 2 of the information processing device 1 derives the cup CE angle as the angle formed by a line connecting the center of rotation (COR) of the cup 81 assumed to be placed in a state in which the pelvis has been removed by the bone excavation reamer 7 to the intersection of the outer edge of the cup 81 and the white line (acetabular sourcil) of the pelvic load surface, and a line connecting the lower ends of the left and right teardrops on the pelvis, which is perpendicular to the pelvic reference line (Y-axis).

[0040] The processing unit 2 of the information processing device 1 derives an internal plate line in the perspective image based on the internal plate corresponding points (landmarks) output by the learning model 101 (S106). The learning model 101 outputs a tomographic image in which the internal plate corresponding points (landmarks) are superimposed on the input tomographic image (e.g., CT image). The internal plate corresponding points are points located on the internal plate line that indicate the maximum cutting point when cutting the pelvis, i.e., the excavation limit line that must not be exceeded. The processing unit 2 of the information processing device 1 extracts each point (landmark) shown in each tomographic image and forms (connects) a line connecting these extracted points (landmarks), thereby deriving (identifying) the internal plate line.

[0041] FIG. 7 is an explanatory diagram of the inner plate line. In the illustration of this embodiment, The diagram shows (three in this example) tomographic images, each of which has a correspondence relationship (same Y coordinate) with a fluoroscopic image (X-ray image) at multiple locations in the Y-axis direction (the vertical direction of the human body). Each of these tomographic images is output from the learning model 101 (inner plate line model), and each tomographic image has a circle superimposed on it indicating an inner plate corresponding point (landmark). The processing unit 2 of the information processing device 1 uses each of the tomographic images on which the inner plate corresponding points (landmarks) are superimposed to identify multiple inner plate corresponding points (landmarks) in the fluoroscopic image, and forms an inner plate line by connecting the identified multiple inner plate corresponding points.

[0042] The processing unit 2 of the information processing device 1 may generate (reconstruct) a three-dimensional medical image including the pelvis and hip joints of patient K using volume data constructed using these multiple tomographic images. In addition, the processing unit 2 of the information processing device 1 may set a three-dimensional internal body coordinate system in the tomographic image or three-dimensional medical image including the pelvis, etc. of patient K. In this internal body coordinate system, the processing unit 2 of the information processing device 1 may define, for example, a line connecting the lower ends of two teardrops on the pelvis as a pelvic reference line, define the horizontal direction with respect to the pelvic reference line (left-right direction with respect to the human body) as the X-axis, the vertical direction with respect to the pelvic reference line (up-down direction with respect to the human body) as the Y-axis, and the direction perpendicular to both the X-axis and Y-axis (front-back direction with respect to the human body) as the X-axis, and perform various arithmetic operations. The processing unit 2 of the information processing device 1 may normalize, i.e., align, the internal body coordinate systems between the acquired fluoroscopic images (X-ray images) and the acquired tomographic images (CT images) using shape recognition results of the same internal body part (e.g., teardrop) included in both the acquired fluoroscopic images (X-ray images) and the acquired tomographic images (CT images). As a result, the processing unit 2 of the information processing device 1 applies the internal body coordinate system set in multiple tomographic images (or reconstructed 3D medical images) to the fluoroscopic images (X-ray images) captured in real time during surgery, and derives an internal body line connecting multiple internal body landmarks in the fluoroscopic images (X-ray images) based on internal body landmarks identified in each tomographic image. Furthermore, the processing unit 2 of the information processing device 1 can also identify the current position of the bone drilling reamer 7 included in the fluoroscopic images (X-ray images) in the internal body coordinate system, which is a common coordinate system with the tomographic images (CT images), by converting the image coordinate system of the fluoroscopic images into the internal body coordinate system.

[0043] The processing unit 2 of the information processing device 1 derives a required line in the fluoroscopic image according to the radius of the cup 81 and the target CE angle (S107). The radius and target CE angle of the cup 81 of the artificial hip joint 8 to be inserted into the patient K are stored in the storage unit 3 as various values ​​determined in the preoperative plan. The processing unit 2 of the information processing device 1 derives a required line in the fluoroscopic image according to the radius of the cup 81 and the target CE angle, based on a white line (Acetabular sourcil) formed by the inner edge of the pelvic load surface included in the fluoroscopic image. The sourcil can be identified (position grasped) by performing shape recognition processing on the fluoroscopic image. The required reach line indicates the minimum cutting point when cutting the pelvis of patient K to place the cup 81.

[0044] 8 is an explanatory diagram of the required line. The memory unit 3 of the information processing device 1 stores various values ​​(parameters) determined in a preoperative plan when performing treatment for the artificial hip joint 8, such as a minimum CE angle (e.g., 10 degrees) set in advance to stabilize the cup 81 to be placed, and the radius of the cup 81 to be implanted. The processing unit 2 of the information processing device 1 acquires the various values ​​determined in the preoperative plan by referring to the memory unit 3.

[0045] The processing unit 2 of the information processing device 1 defines the center (x, y) of a cup 81, which is assumed to be placed in a bone cavity (installation space) excavated by the bone excavation reamer 7, and the intersection (X, Y) of the center (x, y) with the white line (acetabular sourcil) formed by the inner edge of the pelvic load surface when excavated. In this case, the center (x, y) of the cup 81 is expressed by the following equation using the intersection (X, Y), the radius (r) of the cup 81, and the cup CE angle (Θ): The X coordinate (x) of the center of the cup 81 is calculated by subtracting the value obtained by multiplying the radius (r) of the cup 81 by the sine of the cup CE angle (Θ) from the X coordinate (X) of the intersection (x) (x = Xr * sin Θ). The Y coordinate (y) of the center of the cup 81 is calculated by subtracting the value obtained by multiplying the radius (r) of the cup 81 by the cosine of the cup CE angle (Θ) from the Y coordinate (Y) of the intersection (y) (y=Yr*cosΘ). In this way, the center (x,y) of the cup 81 corresponds one-to-one to the intersection (X,Y). When the intersection (X,Y) is changed, that is, when the acetabular sourcil formed by the inner edge of the pelvic load surface during drilling is changed in accordance with the drilling depth by the bone drilling reamer 7, the center (x,y) of the cup 81 also changes. In this way, when the intersection (X,Y) is changed, the cup CE angle (Θ) must exceed the minimum CE angle (10 degrees) (Θ>10), which is a requirement for an implant. The processing unit 2 of the information processing device 1 calculates the intersection (X,Y) within a range that satisfies the requirement (Θ>10). Determine the range of variation within which (X,Y) can be varied.

[0046] The processing unit 2 of the information processing device 1 uses the above equation to identify the area where the center (x, y) of the cup 81 can be located, based on the range of variation of the determined intersection point (X, Y). By taking into account the radius (r) of the cup 81 and the center (x, y) of the cup 81 determined according to the varied intersection point (X, Y), the processing unit 2 of the information processing device 1 derives the edge line within the range of possible values ​​(x', y') indicating each point on the outer edge of the cup 81 as a line (required line) that must be touched by the outer edge of the bone-excavating reamer 7 (the outer edge of the hemispherical tip that performs excavation) to satisfy the requirement (Θ > 10). In deriving the required line, the processing unit 2 of the information processing device 1 selects an arbitrary point (x1, y1) on the floor of the pelvis before excavation with the bone-excavating reamer 7, and defines the angle of the X-axis with respect to the line connecting the center (x, y) of the cup 81 and the arbitrary point as α. The coordinates of the arbitrary point are (x+rcosα, y+rsinα). Then, the processing unit 2 of the information processing device 1 may derive the required line so that the point (x+rcosα, y+rsinα) taking into account the radius (r) of the cup 81 for the center (x, y) of the cup 81 corresponding to the changed intersection point (X, Y) is greater than the coordinates (x1, y1) of the arbitrary point (x+rcosα>x1 and y+rsinα>y2). In other words, the processing unit 2 of the information processing device 1 derives the required line as the range (x', y') that the center (x, y) of the cup 81 that satisfies the conditional expressions "(x+rcosα>x1 and y+rsinα>y2) and "cup CE angle; Θ>10" can take, that is, the edge of the range that can be taken as the outer edge of the cup 81 by taking into account the radius (r) of the cup 81. The required reach line thus derived corresponds to the inner line in the reach area (the inner side on the near side in the excavation direction of the bone excavation reamer 7).

[0047] The processing unit 2 of the information processing device 1 derives an upper limit line in the fluoroscopic image according to the center of the femoral head on the healthy side (S108). The processing unit 2 of the information processing device 1 recognizes the femoral head on the healthy side included in the fluoroscopic image, and derives an upper limit line in the fluoroscopic image according to the center of the femoral head on the healthy side. The upper limit line indicates the limit of uppering that the femoral head center on the affected side (the center of the femoral head ball of the artificial hip joint 8) can take relative to the center of the femoral head on the healthy side.

[0048] FIG. 9 is an explanatory diagram of the upper limit line. In the illustration of this embodiment, four diagrams ((1) to (4)) are shown for reference. These diagrams are shown as follows. Diagram (1) shows a line parallel to the X-axis passing through y' = y2 + 10 mm + cup radius mm (cup edge upper limit line). Diagram (2) shows a line parallel to the X-axis passing through y = y2 + 10 mm (cup center upper limit line). Diagram (3) shows a line parallel to the X-axis passing through the center of the healthy femoral head (x2, y2). Diagram (4) shows a pelvic reference line (X-axis) connecting the lower end of the teardrop scar. Patient K who is to undergo treatment related to an artificial hip joint 8 is generally expected to undergo treatment related to the artificial hip joint 8 (implant) on either the left or right leg, and in this case, the affected side on which treatment related to the artificial hip joint 8 is to be performed is referred to as the affected side, and the healthy side on which treatment related to the artificial hip joint 8 is not to be performed is referred to as the healthy side.

[0049] The processing unit 2 of the information processing device 1 recognizes the shape of the healthy femoral head included in the acquired fluoroscopic image (X-ray image) using an object detection model such as YOLO or edge detection, and derives the healthy femoral head center by calculating the center of curvature using multiple points located on the outer edge based on the arc-shaped outer edge of the femoral head. The processing unit 2 of the information processing device 1 derives an upper limit line (line segment (2)) for the cup center (the femoral head center of the affected implant) that is within a predetermined value (e.g., 10 mm) in the positive direction (upward) on the Y axis of a line (line segment (3)) that passes through the derived healthy femoral head center (x2, y2) and is parallel to the X axis (line segment (4)). The upper limit line for the cup center (the femoral head center of the affected implant) serves as a reference line for determining the cup margin upper limit line (line segment (1)), i.e., the upper limit line (line segment (1)) in the reach area (strike zone).

[0050] The center of the bone on the affected side where the treatment for the artificial hip joint 8 is performed corresponds to the center (x, y) of the cup 81 to be placed. The Y-axis indicates the upper limit direction in the human body, and the positive direction on the Y-axis indicates upward. The Y coordinate (y) of the center (x, y) of the cup 81 to be placed is set such that it does not exceed, for example, 10 mm upward (y < y2 + 10 [mm]) with respect to the Y coordinate (y2) of the center of the bone on the healthy side. Therefore, the processing unit 2 of the information processing device 1 derives an upper limit line (line segment (2)) of the cup center (the center of the bone of the implant on the affected side), which passes through a point moved upward (along the positive direction of the Y-axis) by the upper limit value (for example, 10 mm) with respect to the Y coordinate (y2) of the center of the bone on the healthy side and is perpendicular to the Y coordinate. Further, a cup edge upper limit line (line segment (1)), which is parallel to the upper limit line (line segment (2)) of the cup center (the center of the bone of the implant on the affected side) (that is, parallel to the X-axis) and is located upward (the positive direction on the Y-axis) by the cup radius, that is, the upper limit line (line segment (1)) in the reach area (strike zone) is determined (derived). By deriving the upper limit line (line segment (1): cup edge upper limit line) in the reach area (strike zone) based on the center of the bone on the healthy side in this way, a guideline (the upper limit line in the excavation of the pelvis) when the center of the bone on the affected side (the center of the femoral head of the artificial hip joint 8) is placed can be provided to a doctor or the like.

[0051] The processing unit 2 of the information processing device 1 derives the reach area (strike zone) based on the derived inner plate line, must-reach line, and upper limit line (S109). The processing unit 2 of the information processing device 1 derives the area surrounded by these lines as the reach area (strike zone) based on the derived upper limit line, inner plate line, and must-reach line.

[0052] FIG. 10 is an explanatory diagram of the strike zone. The strike zone indicates the area where stable fixation of the cup 81 is ensured if a portion of the cup 81 is in contact with the strike zone when the cup 81 is placed. In the strike zone, the upper limit line and the inner plate line indicate limit lines that must not be exceeded when drilling with the bone drilling reamer 7. By superimposing the strike zone thus derived on the fluoroscopic image, the positional relationship between the current position of the bone drilling reamer 7 and the area that the bone drilling reamer 7 should reach by drilling (the strike zone) can be provided to the physician in real time by the bone drilling reamer 7 included in the fluoroscopic image displayed in real time.

[0053] The processing unit 2 of the information processing device 1 derives various current actual measurement values, target values, and differences between them (S110). Using the fluoroscopic image, the processing unit 2 of the information processing device 1 derives actual measurement values ​​of the cup 81 that is expected to be placed in a state in which the pelvis has been reamed with the bone drilling reamer 7. The target CE angle and the target distance between the outer edge of the cup 81 and the inner plate line are set in advance as target values ​​and are stored in the memory unit 3 of the information processing device 1 as various values ​​determined in the preoperative plan when performing treatment related to the artificial hip joint 8. The processing unit 2 of the information processing device 1 acquires the various values ​​determined in the preoperative plan by referring to the memory unit 3.

[0054] The processing unit 2 of the information processing device 1 uses the acquired fluoroscopic image to identify the position and inclination of the cup 81 when it is assumed that the cup 81 will be placed in a state where the pelvis has been cut by the bone drilling reamer 7, based on the current position and inclination of the bone drilling reamer 7 (the outer edge of the hemispherical tip that performs drilling) in the pelvis of the patient K. The inclination includes, for example, the anteversion angle and the abduction angle of the cup 81.

[0055] FIG. 11 is an explanatory diagram of the anteversion angle and abduction angle of the cup 81 (assumed position when the pelvis is cut by the bone-excavating reamer 7). The processing unit 2 of the information processing device 1 identifies three points (A, A', B) of an ellipse by shape recognition processing. In this case, if the angle formed by A-A' and A'-B is X in the fluoroscopic image, the anteversion angle (Anteversion angle) is expressed as "sin^(-1)*tanX". The processing unit 2 identifies the anteversion angle and abduction angle of the cup 81 (assumed position when the pelvis is cut by the bone-excavating reamer 7) in the fluoroscopic image. The shape and current position (of the outer edge of the hemispherical tip that performs the excavation) may be recognized by calculation using the image coordinate system in the perspective image, and the forward tilt angle and abduction angle may be calculated.

[0056] The processing unit 2 of the information processing device 1 may derive a target target forward tilt angle and a target abduction angle based on a preset target CE angle. The processing unit 2 of the information processing device 1 may derive the target forward tilt angle and the target abduction angle by referring to a table (target CE angle table) in which the values ​​of the target forward tilt angle and the target abduction angle are associated with each value of the target CE angle. Various lookup tables, such as the target CE angle table, that the processing unit 2 of the information processing device 1 refers to when performing various calculation processes are stored in the storage unit 3.

[0057] The processing unit 2 of the information processing device 1 outputs the derived various support information (arrival area, differences between actual measurement values ​​and target values) superimposed on and accompanying the fluoroscopic image (S111). The support information includes, for example, the arrival area and differences between actual measurement values ​​and target values. The processing unit 2 of the information processing device 1 superimposes the derived arrival area on the fluoroscopic image and outputs it to the display device 41. The processing unit 2 of the information processing device 1 further associates various actual measurement values, target values, and differences between these actual measurement values ​​and target values ​​with the fluoroscopic image and outputs it to the display device 41. When outputting this information, the processing unit 2 of the information processing device 1 may generate and output screen data that constitutes a display screen (support information superimposed screen). The processing unit 2 of the information processing device 1 compares the derived actual measurement values ​​(cup CE angle, actual measured distance between the outer edge of the cup 81 and the inner plate line, anteversion angle, abduction angle) with the corresponding target values ​​(target CE angle, target distance between the outer edge of the cup 81 and the inner plate line, target anteversion angle, target abduction angle), and displays them in association with the fluoroscopic image (for example, on a sub-screen or in a separate frame). This allows the doctor or the like to efficiently grasp the differences between the various actual measurement values ​​at the current time and the target values.

[0058] The processing unit 2 of the information processing device 1 extracts a tomographic image including the current position of the bone drilling reamer 7 from among multiple tomographic images of the pelvis of patient K (S112). The processing unit 2 of the information processing device 1 identifies the current position of the bone drilling reamer 7 (the outer edge of the hemispherical tip that drills) in the pelvis of patient K by performing shape recognition on the acquired fluoroscopic image using, for example, an object detection model such as YOLO or edge detection. The fluoroscopic image is aligned with the tomographic image (CT image, etc.), and the processing unit 2 of the information processing device 1 can apply the internal body coordinate system defined in the tomographic image to the fluoroscopic image.

[0059] The processing unit 2 of the information processing device 1 may acquire the current position of the bone drilling reamer 7 (the outer edge of the hemispherical tip that performs drilling) included in the fluoroscopic image by identifying it in the internal body coordinate system defined by the tomographic image. The storage unit 3 of the information processing device 1 stores a plurality of tomographic images (CT images, etc.) of the patient K taken by the CT device 61 before pelvic drilling with the bone drilling reamer 7. These multiple tomographic images (CT images, etc.) are images sliced ​​in the Y-axis direction, i.e., the anterior-posterior direction of the human body, and each tomographic image (CT image, etc.) is represented by the X-axis (left-right direction of the human body) and the Z-axis (anterior-posterior direction of the human body).

[0060] The processing unit 2 of the information processing device 1 extracts one or more tomographic images (e.g., CT images) including the current position of the bone drilling reamer 7, according to the fluoroscopic image. The processing unit 2 of the information processing device 1 extracts a tomographic image (e.g., CT image) corresponding to the center of the bone drilling reamer 7 (the outer edge of the hemispherical tip that performs drilling), i.e., the Y-coordinate (y) of the center (x, y) of the cup 81 at the location where the cup 81 is expected to be placed in the current drilling state. Alternatively, since each individual tomographic image is associated with a Y-axis coordinate and an area (range) on the XZ plane corresponding to the tomographic image using, for example, an internal body coordinate system, the processing unit 2 of the information processing device 1 may extract a tomographic image corresponding to the current position of the bone drilling reamer 7 in, for example, the internal body coordinate system.

[0061] The processing unit 2 of the information processing device 1 superimposes a cup periphery diagram, which is a diagram of the cup 81 assumed to be positioned when the pelvis has been cut by the bone-drilling reamer 7, on the extracted tomographic image and outputs the diagram (S113). The processing unit 2 of the information processing device 1 superimposes a semicircular figure (cup periphery diagram) indicating the current position of the bone-drilling reamer 7 (the outer edge of the hemispherical tip that performs drilling) on ​​one or more extracted tomographic images (CT images, etc.). The processing unit 2 of the information processing device 1 may further use the fluoroscopic image to derive the installation angle and depth of the cup 81 at the location where the cup 81 is assumed to be placed in the current drilling state, and output the diagram by further superimposing it on the tomographic image (CT image, etc.). In this way, the processing unit 2 of the information processing device 1 extracts a corresponding tomographic image (e.g., a CT image) according to the current position of the bone drilling reamer 7 identified in the fluoroscopic image (the location where the cup 81 is expected to be placed in the current drilling state), and outputs the tomographic image by superimposing on it an outline drawing (a semicircular graphic object) of the cup 81 where it is expected to be placed, and the installation angle and depth of the cup 81. This makes it possible to provide a doctor or the like with information to support three-dimensional intraoperative recognition, including information about the Z axis (the front-to-back direction in the human body).

[0062] 12 is an explanatory diagram illustrating an example of a support information display screen (support information superimposed screen). As a result of the various processes described above, the processing unit 2 of the information processing device 1 generates screen data that constitutes the support information superimposed screen and outputs the screen data to the display device 41. The support information superimposed screen includes a perspective image display area, a tomographic image display area, and a support information display area.

[0063] The fluoroscopic image display area displays a fluoroscopic image (X-ray image) of the pelvis of patient K captured in real time during surgery while the pelvis is being cut with the bone drilling reamer 7, with the reachable area superimposed on the fluoroscopic image. The tomographic image display area displays a tomographic image (CT image) including the current position of the bone drilling reamer 7 identified in the fluoroscopic image, with a diagram of the outer edge of cup 81 (semicircular graphic object) superimposed on the tomographic image, assuming the placement of cup 81 after the pelvis has been cut with the bone drilling reamer 7. The reachable area may also be superimposed on the tomographic image.

[0064] The support information display area displays in list form the anteversion angle, abduction angle, cup CE angle, and remaining distance from the inner plate line of the assumedly positioned cup 81 when the pelvis has been removed with the bone excavation reamer 7. For the anteversion angle, abduction angle, cup CE angle, and remaining distance from the inner plate line of the cup 81, the actual measured values ​​calculated based on the current position and inclination of the bone excavation reamer 7 identified by shape recognition processing on the fluoroscopic image, the values ​​(target values) determined in the preoperative plan, and the difference between the actual measured values ​​and the target values ​​are displayed in list form.

[0065] 13 is an explanatory diagram illustrating an example of a display screen (distortion correction screen) for support information. As a result of the various processes described above, the processing unit 2 of the information processing device 1 generates screen data constituting the distortion correction screen and outputs the screen data to the display device 41. The distortion correction screen includes a display area before distortion correction and a display area after distortion correction. As described above, the distortion correction screen displays a perspective image before distortion correction (with support information superimposed) and a perspective image after distortion correction (with support information superimposed) in comparison, and therefore can present information indicating the reachable area (strike zone) before and after distortion correction to a doctor or the like, thereby providing the doctor with useful information when performing a procedure.

[0066] The distortion correction screen may be output as a separate screen from the support information superimposed screen described above, or may be included in the support information superimposed screen. When the distortion correction screen is output as a separate screen from the support information superimposed screen, the display device 41 displaying the distortion correction screen and the display device 41 displaying the support information superimposed screen may be separate display devices 41. In this case, two display devices 41, one displaying the distortion correction screen and the other display device 41 displaying the support information superimposed screen, are connected to the information processing device 1.

[0067] The image displayed in the perspective image display area on the support information superimposed screen (the perspective image with the reachable area superimposed) is displayed in the pre-distortion correction display area. Furthermore, the image displayed in the pre-distortion correction display area (the pre-correction image) displays a grid of auxiliary lines made up of a plurality of curves that are used for distortion correction.

[0068] The distortion-corrected image (a perspective image with the reach area superimposed) is displayed in the distortion-corrected display area. Furthermore, the image (corrected image) displayed in the distortion-corrected display area displays the auxiliary lines used in the distortion correction process in a state where they have been transformed into straight lines in accordance with the correction. As a result, the reach area (strike zone) is also displayed with its shape (the area in the pelvis) changed in accordance with the distortion correction.

[0069] When performing distortion correction, the processing unit 2 of the information processing device 1 may, for example, attach a template including straight lines under the fluoroscopic image (image before correction) to perform distortion correction processing. In this case, the processing unit 2 of the information processing device 1 is a device that converts X-rays that have passed through the human body into a digital image, and may use a flat panel with a distortion correction function.

[0070] Alternatively, the processing unit 2 of the information processing device 1 may perform distortion correction processing on a fluoroscopic image (an image before correction) in accordance with distortion determined based on the X-ray imaging characteristics of an X-ray device 62 or the like. The X-ray device 62 that captures a fluoroscopic image (X-ray image) has various characteristics depending on the model or type, and it is also possible to identify distortion corresponding to the amount of distortion in the captured fluoroscopic image (X-ray image). The storage unit 3 of the information processing device 1 may store parameters such as distortion corresponding to each of the X-ray devices 62, and the processing unit 2 of the information processing device 1 may perform distortion correction using the parameters such as distortion corresponding to the X-ray device 62 or the like. In this case, it is not necessary to use a flat panel, which is a dedicated device for performing distortion correction by hardware processing, and the distortion-corrected fluoroscopic image can be displayed using a relatively inexpensive display (a display device without a distortion correction function by hardware processing).

[0071] According to this embodiment, the processing unit 2 of the information processing device 1 acquires a fluoroscopic image (X-ray image) of a patient K undergoing treatment for an artificial hip joint 8. The fluoroscopic image is, for example, an X-ray image. When a bone drilling reamer 7 is used to form a bony depression (installation space) that will serve as a pelvic receptacle for installing a cup 81 of the artificial hip joint 8 (drilling the pelvis) as part of the treatment for the artificial hip joint 8, the fluoroscopic image (X-ray image) is captured as a moving image in real time, and the processing unit 2 of the information processing device 1 sequentially acquires the fluoroscopic images (X-ray images) captured in real time. The processing unit 2 of the information processing device 1 recognizes (derives) an inner plate line indicating the maximum cutting point when cutting the pelvis to install the cup 81 including the artificial hip joint 8, and a required line (a line to be exceeded) indicating the minimum cutting point when cutting the pelvis to install the cup 81, in the pelvis of the patient K in the acquired fluoroscopic image. Furthermore, the processing unit 2 of the information processing device 1 derives the cup CE angle when it is assumed that the cup 81 will be positioned in a state where the pelvis has been reamed with the bone-excavating reamer 7 (in real time). The processing unit 2 of the information processing device 1 superimposes the recognized (derived) inner plate line and necessary reach line on the fluoroscopic image and displays the derived cup CE angle in association with the fluoroscopic image, for example, on a sub-screen, thereby providing useful support information to a doctor performing a procedure related to the artificial hip joint 8 on patient K. The processing unit 2 of the information processing device 1 recognizes the shape of the outer edge of the bone-excavating reamer 7 (the outer edge of the hemispherical tip that performs the excavation) included in the fluoroscopic image (X-ray image) using, for example, an object detection model such as YOLO or edge detection. The processing unit 2 of the information processing device 1 regards the outer edge of the bone-excavating reamer 7 as the outer edge of the cup 81 and derives the cup CE angle when it is assumed that the cup 81 will be positioned in a state where the pelvis has been reamed with the bone-excavating reamer 7. The processing unit 2 of the information processing device 1 derives the cup CE angle as the angle formed by a line connecting the center of rotation (COR) of the cup 81, which is assumed to be positioned when the pelvis has been cut by the bone-drilling reamer 7, to the intersection of the outer edge of the cup 81 and the white line (Acetabular sourcil) of the pelvic load surface, and a line connecting the lower ends of the left and right teardrops on the pelvis, which is perpendicular to the pelvic reference line (Y-axis).The processing unit 2 of the information processing device 1 performs shape recognition of the lower ends of the left and right teardrops included in the fluoroscopic image (X-ray image) using, for example, an object detection model such as YOLO or edge detection processing, and identifies a pelvic reference line passing through these lower ends of the left and right teardrops. The processing unit 2 of the information processing device 1 identifies a vertical line that is perpendicular to the identified pelvic reference line and passes through the center of ream (COR) of the cup 81, which is assumed to be placed in a state where the pelvis has been reamed with the bone-reaming reamer 7. The processing unit 2 of the information processing device 1 derives the cup CE angle by calculating the angle between the vertical line passing through the center of ream (COR) of the cup 81 and a line connecting the center of the cup 81 and the intersection of the outer edge of the cup 81 and the acetabular sourcil of the pelvic load surface. The processing unit 2 of the information processing device 1 acquires fluoroscopic images captured in real time and derives the cup CE angle based on the outer edge of the bone-reaming reamer 7 included in the acquired fluoroscopic images. This makes it possible to track the current position of the bone drilling reamer 7, i.e., the drilling state by the bone drilling reamer 7, and provide the surgeon with the cup CE angle when the cup 81 is currently placed at any time. In other words, it is possible to configure an intraoperative support system S that is capable of performing real-time intraoperative evaluation of the cup 81 placement and coverage rate.

[0072] According to this embodiment, one or more tomographic images of the pelvis of patient K, such as CT images or MRI images, are captured before drilling the pelvis of patient K with the bone drilling reamer 7. The processing unit 2 of the information processing device 1 acquires these multiple tomographic images and stores them in the memory unit 3. The memory unit 3 of the information processing device 1 stores an entity file of a learning model 101 (inner plate line model) that has been trained to output position information (landmarks) of the inner plate when a tomographic image is input. The processing unit 2 of the information processing device 1 inputs the acquired tomographic image (e.g., a CT image) to the learning model 101 (inner plate line model), which then outputs the position information of the inner plate superimposed on the tomographic image. Each piece of position information of the inner plate may be output as a point (landmark). By outputting multiple points (landmarks) on the tomographic image, a line connecting these multiple points (landmarks) may be formed (connected) to identify the inner plate line. In this way, when a tomographic image is input, by using a learning model 101 (inner plate line model) that has been trained to output position information (landmarks) of the inner plate, it is possible to efficiently identify (derive) the inner plate line that indicates the maximum cutting point when cutting the pelvis to position the cup 81 containing the artificial hip joint 8. The processing unit 2 of the information processing device 1 aligns the tomographic image (CT image, etc.) and the fluoroscopic image (X-ray image) based on the same internal body part (for example, teardrop) identified using shape recognition, etc., and defines these two images (tomographic image and fluoroscopic image) in the same internal body coordinate system, and can perform various calculation processes such as superimposing the images. The processing unit 2 of the information processing device 1 aligns the internal body coordinate value (internal body coordinate value) of the fluoroscopic image so that the internal body coordinate value (internal body coordinate value) of the inner plate line derived using the tomographic image (CT image, etc.) is the same as the internal body coordinate value of the inner plate line shown in the tomographic image. The fluoroscopic image (X-ray image) displayed in real time during surgery shows the bone drilling reamer 7 at the current time, and the internal plate line is superimposed on the image, so that the doctor or the like operating the bone drilling reamer 7 can see the maximum cutting point when cutting the pelvis, that is, the internal plate line connected at the limit point that should not be exceeded. (Excavation limit line) can be effectively presented.

[0073] The embodiments disclosed herein are to be considered in all respects as illustrative and not restrictive. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims.

[0074] Multiple claims may be combined with each other regardless of the form of reference. Multiple dependent claims may be included in the claims, depending on multiple claims. Multiple dependent claims may be included in a multiple dependent claim. If multiple dependent claims are not included in a multiple dependent claim, this does not limit the number of multiple dependent claims that are included in a multiple dependent claim. [Explanation of symbols]

[0075] S Intraoperative Support System K patient 1. Information processing equipment 2 Processing section 3 Storage section M Recording medium P Program (Program Product) 4 Input / Output Interface 41 Display device 5. Communications Department 101 Learning Model (Inner Board Line Model) 61 CT device (tomographic imaging device) 62 X-ray equipment (fluoroscopic imaging equipment) 7 Bone Drilling Reamer 8. Artificial hip joints 81 cups

Claims

1. On the computer, obtaining a fluoroscopic image of a patient undergoing a hip prosthesis procedure using a bone drilling reamer; In the acquired fluoroscopic image of the patient's pelvis, an inner plate line indicating the maximum cutting point when cutting the pelvis to place a cup included in the artificial hip joint is recognized. A program that executes a process.

2. A line indicating the minimum cutting point when cutting the pelvis to place the cup is recognized in the patient's pelvis. The program according to claim 1.

3. The recognized inner plate line and the required line are superimposed on the perspective image. The program according to claim 2.

4. acquiring a tomographic image of the patient's pelvis; deriving position information of the inner plate by inputting the acquired tomographic image into a learning model that has been trained to output position information of the inner plate when a tomographic image is input; The inner plate line in the perspective image is recognized based on the output group of position information of the plurality of inner plates. The program according to claim 2.

5. Based on the femoral head center of the healthy side where no treatment related to the artificial hip joint is performed, an upper limit line is derived for the femoral head center of the affected side where treatment related to the artificial hip joint is performed; Derive a reachable region based on the derived upward limit line, the inner plate line, and the required reachable line; The derived reachable area is superimposed on the perspective image. The program according to claim 2.

6. The required line is derived based on a target CE angle that is predetermined to stabilize the cup that is placed, an intersection point between the outer edge of the cup and the inner edge of the load surface of the patient's pelvis, which serves as a reference when ensuring the target CE angle, and the radius of the cup. The program according to claim 2.

7. Deriving the actual measured distance between the outer edge of the cup and the inner plate line when assuming the placement of the cup in a state where the patient's pelvis has been cut by the bone drilling reamer, deriving a target distance between the outer edge of the cup and the inner plate line, the target distance being preset as a target value; The calculated actual distance is compared with the target distance and output. The program according to claim 2.

8. deriving an anteversion angle and an abduction angle of the cup when assuming the placement of the cup in a state in which the patient's pelvis has been cut with the bone drilling reamer; Output the derived anteversion angle and abduction angle The program according to claim 2.

9. Deriving a target forward tilt angle and a target abduction angle according to a target CE angle that is preset as a target value; The derived target forward tilt angle and the target abduction angle are combined and output. The program according to claim 2.

10. The derived anteversion angle and abduction angle of the cup are compared with the target anteversion angle and the target abduction angle and output. The program according to claim 9.

11. obtaining a current position of the bone reamer in the patient's pelvis based on the obtained fluoroscopic image; extracting a tomographic image including the current position of the bone drilling reamer from the plurality of tomographic images of the patient's pelvis; An outline of the cup when the placement of the cup is assumed in a state where the patient's pelvis has been cut by the bone drilling reamer is superimposed on the extracted tomographic image and output. The program according to claim 4.

12. Obtaining a fluoroscopic image of a patient undergoing a procedure related to an artificial hip joint using a bone drilling reamer; In the acquired fluoroscopic image of the patient's pelvis, an inner plate line indicating the maximum cutting point when cutting the pelvis to place a cup included in the artificial hip joint is recognized. An information processing method in which processing is performed by a computer.

13. An acquisition unit for acquiring fluoroscopic images of a patient undergoing a procedure related to an artificial hip joint using a bone drilling reamer; an inner plate line recognition unit that recognizes an inner plate line indicating the maximum cutting point when cutting the pelvis to place a cup included in the artificial hip joint in the patient's pelvis in the acquired fluoroscopic image; An information processing device comprising:

Images