Deep learning-based surgery plan system for reverse shoulder arthroplasty, and driving method thereof

Abstract

The method for generating a deep learning-based surgery plan, related to an embodiment of the present invention may include: a first step of acquiring X-ray image information; a second step of performing preprocessing for analysis of a predetermined region of the X-ray image information; a third step of analyzing the preprocessed X-ray image by using a convolutional neural network and extracting feature information from the result of the analysis and learning same; a fourth step of repeating the first step to the third step and generating template data for the predetermined region, on the basis of the repeatedly learned feature information; a fifth step of obtaining first X-ray image information of a user; a sixth step of extracting first feature information related to the user from the feature information by using the first X-ray image information and the template data; and a seventh step of generating a surgery plan related to the user, on the basis of the first feature information.

Description

Deep learning-based surgical planning system for retrograde total shoulder arthroplasty and its operation method

[0001] The present invention relates to a surgical planning system for reverse shoulder arthroplasty based on deep learning and a method for operating the same. Specifically, the present invention proposes a surgical planning system that automatically establishes a surgical plan before surgery for reverse shoulder arthroplasty, and a comprehensive surgical planning system for reverse shoulder arthroplasty that can automatically design and produce a patient-specific surgical guide (Patient Specific Instrument) for performing a fast and accurate surgery according to the established surgical plan.

[0002] Performing reverse total shoulder arthroplasty can present several challenges and potential problems, and without preoperative planning, it can be difficult for the surgeon to accurately determine the appropriate size and position of the prosthetic components.

[0003] Additionally, inadequate implant sizing can lead to instability, limited range of motion, or joint inconsistency, which can lead to suboptimal results and potential complications, as well as challenges in identifying intraoperative landmarks, determining appropriate bone resection, and optimizing soft tissue balance.

[0004] Furthermore, this may result in longer surgical times and increased risk of complications, patient discomfort, and surgical fatigue.

[0005] Therefore, the need for methods and systems that can solve this problem is increasing.

[0006] The present invention proposes a surgical planning system that automatically establishes a surgical plan prior to surgery for retrograde total shoulder arthroplasty that can solve the aforementioned problems.

[0007] The present invention proposes a comprehensive reverse total shoulder arthroplasty surgical planning system that can automatically design and produce a patient-specific surgical guide (Patient Specific Instrument) for performing a fast and accurate surgery according to an established surgical plan.

[0008] The deep learning within the system applied to the present invention proposes a system that determines the cutting level of the acromion and recommends products from specific companies to proceed with templating.

[0009] Furthermore, the present invention proposes a system and method for designing a patient-specific surgical guide using a Boolean algorithm, thereby using 3D printing to produce a guide that can be used in surgery without a separate design work.

[0010] The technology proposed by the present invention can be summarized as follows.

[0011] 1) Deep learning-based implant recommendation and resection range determination

[0012] 2) 2D and 3D medical image processing technology

[0013] Meanwhile, the technical tasks to be achieved in the present invention are not limited to the technical tasks mentioned above, and other technical tasks not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.

[0014] A deep learning-based surgical plan generation method according to an example of the present invention for realizing the above-described task may include: a first step of acquiring X-ray image information; a second step of performing preprocessing to analyze a pre-designated area of ​​the X-ray image information; a third step of analyzing the pre-processed X-ray image using a convolutional neural network (CNN) and extracting and learning feature information from the analysis result; a fourth step of repeating the first to third steps and generating template data for the pre-designated area based on the iteratively learned feature information; a fifth step of acquiring first X-ray image information of a user; a sixth step of extracting first feature information related to the user from the feature information using the first X-ray image information and the template data; and a seventh step of generating a surgical plan related to the user based on the first feature information.

[0015] Additionally, the surgery associated with the user may include a reverse total shoulder arthroplasty, and the pre-designated area may include a shoulder joint area.

[0016] Additionally, the preprocessing in the second step can be applied to enable analysis of anatomical information about the shoulder joint area.

[0017] Additionally, the feature information may include at least one of anatomical structure information related to the user, bone structure information, angle information of a pre-specified joint, muscle and / or tissue analysis information, and information related to the size and position of an implant structure to be applied to the shoulder joint.

[0018] In addition, the generated surgical plan may include at least one of information determining the type, size, and location of the implant to be applied to the shoulder joint, information on the cutting range and cutting angle for applying the reverse shoulder joint replacement surgery, information on prediction of potential complications after the surgery, information on prediction of the user's range of motion related to the shoulder joint area after the surgery, and information on an evaluation score for the suitability of the established surgical plan determined based on pre-stored surgical data.

[0019] In addition, in the first step, a CT image of a previously completed surgery is acquired, and in the third step, a cutting level applied to the completed surgery is measured based on the CT image, a portion of the area related to the completed surgery is extracted from the X-ray image, and the learning can be performed after labeling the preprocessed X-ray image based on the cutting level and the extracted portion of the area.

[0020] In addition, in the fourth step, based on an area selected by an external user from among the X-ray image information, a first shape is automatically determined based on the size of the selected area from among a plurality of pre-stored component shapes, and the template data can be generated based on the first shape.

[0021] In addition, after the 7th step, the method may further include: an 8th step of acquiring CT image information of the user; a 9th step of automatically generating 3D image information by 3D reconstructing the CT image information; and a 10th step of receiving the first X-ray image information and the template data, and generating STL information by using the automatically generated 3D image information, the first X-ray image information, and the template data together; an 11th step of inputting the generated STL data into a 3D printer; and a 12th step of outputting a surgical guide to which the surgical plan has been simulated in advance based on the STL data from the 3D printer.

[0022] In addition, in the ninth step, the PSI size related to the CT image information can be changed using the cutting level on the surgical plan, and the three-dimensional image information can be automatically generated by performing a two-dimensional Boolean operation on the image information with the changed PSI size to fit the user's skeleton.

[0023] Meanwhile, a deep learning-based surgical plan generation device related to another example of the present invention for realizing the above-described task includes: an acquisition unit for acquiring X-ray image information; a preprocessing unit for performing preprocessing to analyze a pre-designated area among the X-ray image information; and a control unit for analyzing the preprocessed X-ray image using a convolutional neural network (CNN) and extracting and learning feature information from the analysis result; wherein after repeating the operations of the acquisition unit, the preprocessing unit, and the control unit, the control unit generates template data for the pre-designated area based on the repeatedly learned feature information, and when the acquisition unit additionally acquires first X-ray image information of the user, the control unit extracts first feature information related to the user among the feature information using the first X-ray image information and the template data, and generates a surgical plan related to the user based on the first feature information.

[0024] The present invention utilizes deep learning to make highly accurate decisions on areas that are difficult for surgeons to determine, such as cutting levels, and can resolve the overall surgical planning by recommending the most suitable artificial joint product for the patient based on images alone, as well as performing templating.

[0025] In addition, by providing designs for patient-tailored surgical guides, they can be produced using 3D printing without separate design work and applied directly to patient surgery, helping to ensure that the surgery can proceed according to the surgical plan.

[0026] In this case, the time required to establish a surgical plan and perform the surgical procedure can be drastically shortened, and the gap in the experience and skills of the surgeon can be reduced, allowing even less experienced surgeons to perform the surgery accurately, and the patient's pain due to the surgery can also be reduced.

[0027] Meanwhile, the effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects that are not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.

[0028] FIG. 1 illustrates an example of a block diagram of a deep learning-based surgical plan generation system in relation to the present invention.

[0029] FIG. 2 illustrates an example of a flowchart explaining a deep learning-based surgical plan generation method in relation to the present invention.

[0030] FIG. 3 illustrates an example of the UI configuration and function of a surgical planning system for retrograde total shoulder arthroplasty in relation to the present invention.

[0031] FIG. 4 illustrates an example of a cutting level determination deep learning model schematic diagram in relation to the present invention.

[0032] FIG. 5 illustrates an example of a templated deep learning model schematic diagram in relation to the present invention.

[0033] FIG. 6 illustrates an example of a Boolean-based PSI automatic design algorithm in relation to the present invention.

[0034] Reverse shoulder arthroplasty is an orthopedic surgical procedure used to treat certain shoulder conditions associated with severe arthritis, irreparable rotator cuff tears, or complex fractures.

[0035] Unlike conventional total shoulder arthroplasty, which replaces the acromion with an artificial ball and the joint space with an artificial socket, reverse total shoulder arthroplasty can reverse the anatomical structure of the shoulder joint.

[0036] Reverse shoulder arthroplasty aims to relieve pain, restore shoulder function, and improve range of motion in individuals with limited mobility or strength due to rotator cuff dysfunction or severe arthritis.

[0037] This procedure bypasses the need for rotator cuff function by primarily utilizing the deltoid muscle.

[0038] The need for a preoperative planning system for reverse total shoulder arthroplasty lies in its ability to improve surgical outcomes, enhance patient safety, and optimize the overall surgical process.

[0039] Applying deep learning to these systems has several advantages.

[0040] Deep learning algorithms can analyze medical images and extract detailed information with a high level of accuracy.

[0041] This allows for precise measurement of anatomical structures, implant size and location, which can improve the accuracy of surgical planning.

[0042] Additionally, by learning from large data sets and previous surgical cases, we can provide personalized treatment plans based on the patient's specific anatomy, pathology, and desired outcomes.

[0043] This individualized approach leads to better surgical outcomes and patient satisfaction.

[0044] In this case, the system can quickly process large amounts of medical image data, enabling faster preoperative planning.

[0045] This saves surgeons valuable time and streamlines surgical workflow, helping to reduce patient wait times and improve hospital efficiency.

[0046] Surgical planning systems that utilize deep learning models can analyze complex patterns in medical images to provide surgeons with useful insights and recommendations.

[0047] This allows for informed decisions about implant selection, surgical technique, and potential complications, reducing the risk of error and improving patient safety.

[0048] Performing reverse total shoulder arthroplasty can present several challenges and potential problems, and without preoperative planning, it can be difficult for the surgeon to accurately determine the appropriate size and position of the prosthetic components.

[0049] Additionally, inadequate implant sizing can lead to instability, limited range of motion, or joint inconsistency, which can lead to suboptimal results and potential complications, as well as challenges in identifying intraoperative landmarks, determining appropriate bone resection, and optimizing soft tissue balance.

[0050] Furthermore, this may result in longer surgical times and increased risk of complications, patient discomfort, and surgical fatigue.

[0051] The present invention proposes a surgical planning system that automatically establishes a surgical plan prior to surgery for retrograde total shoulder arthroplasty that can solve these problems.

[0052] The present invention proposes a comprehensive reverse total shoulder arthroplasty surgical planning system that can automatically design and produce a patient-specific surgical guide (Patient Specific Instrument) for performing a fast and accurate surgery according to an established surgical plan.

[0053] The deep learning within the system applied to the present invention proposes a system that determines the cutting level of the acromion and recommends products from specific companies to proceed with templating.

[0054] Furthermore, the present invention proposes a system and method for designing a patient-specific surgical guide using a Boolean algorithm, thereby using 3D printing to produce a guide that can be used in surgery without a separate design work.

[0055] The technology proposed by the present invention can be summarized as follows.

[0056] 1) Deep learning-based implant recommendation and resection range determination

[0057] 2) 2D and 3D medical image processing technology

[0058] FIG. 1 illustrates an example of a block diagram of a deep learning-based surgical plan generation system in relation to the present invention.

[0059] Referring to FIG. 1, a deep learning-based surgical plan generation system (1) may include an acquisition unit (10) that acquires X-ray image information, a preprocessing unit (20) that performs preprocessing for analysis of a pre-designated area among the X-ray image information, and a control unit (30) that analyzes the preprocessed X-ray image using a convolutional neural network (CNN) and extracts and learns feature information from the analysis results.

[0060] Here, after repeating the operations of the acquisition unit (10), preprocessing unit (20), and control unit (30), the control unit (30) generates template data for the pre-specified area based on the repeatedly learned feature information.

[0061] In addition, when the acquisition unit (10) additionally acquires the user's first X-ray image information, the control unit (30) uses the first X-ray image information and template data to extract the first feature information related to the user from among the feature information, and generates a surgical plan related to the user based on the first feature information.

[0062] Meanwhile, the control unit (30) can obtain the user's CT image information and automatically generate 3D image information by 3D reconstructing the CT image information.

[0063] Additionally, the control unit (30) or the 3D printing unit (2) can receive the first X-ray image information and the template data, and use the automatically generated 3D image information, the first X-ray image information, and the template data together to generate STL information.

[0064] When the generated STL data is input into the 3D printer section (2), the 3D printer section (2) can output a surgical guide that simulates the surgical plan in advance based on the STL data.

[0065] Meanwhile, FIG. 2 illustrates an example of a flowchart explaining a deep learning-based surgical plan generation method in relation to the present invention.

[0066] The surgery involved here may include reverse total shoulder arthroplasty, and the pre-specified area may include the shoulder joint area.

[0067] Referring to Fig. 2, first, a step (S1) of acquiring X-ray image information is performed.

[0068] Afterwards, a step (S2) of performing pre-processing for analysis of a pre-designated area of ​​X-ray image information is performed.

[0069] Here, preprocessing can be applied to enable analysis of anatomical information about the shoulder joint area.

[0070] In addition, a step (S3) is performed in which the preprocessed X-ray image is analyzed using a convolutional neural network (CNN), and feature information is extracted and learned from the analysis results.

[0071] Here, the feature information may include at least one of anatomical structure information related to the user, bone structure information, angle information of a pre-specified joint, muscle and / or tissue analysis information, and information related to the size and position of an implant structure to be applied to the shoulder joint.

[0072] Thereafter, steps 1 to 3 are repeated, and a step (S4) of generating template data for a pre-specified area based on the repeatedly learned feature information is performed.

[0073] Meanwhile, in the first step, if a CT image of a previously completed surgery is acquired together, in the third step, a cutting level applied to the completed surgery is measured based on the CT image, a portion of the area related to the completed surgery is extracted from the X-ray image, and the learning can be performed after labeling the preprocessed X-ray image based on the cutting level and the extracted portion of the area.

[0074] In addition, based on an area selected by an external user among X-ray image information, a first shape may be automatically determined based on the size of the selected area among a plurality of pre-stored component shapes, and the template data may be generated based on the first shape.

[0075] Additionally, a step (S5) of acquiring the user's first X-ray image information is performed.

[0076] Thereafter, a step (S6) of extracting first feature information related to the user from among feature information is performed using the first X-ray image information and template data.

[0077] Additionally, a step (S7) of generating a surgical plan related to the user based on the first feature information is performed.

[0078] The surgical plan generated here may include at least one of information determining the type, size, and location of the implant to be applied to the shoulder joint, information on the cutting range and cutting angle for applying the reverse shoulder joint replacement surgery, information on prediction of potential complications after the surgery, information on prediction of the user's range of motion related to the shoulder joint area after the surgery, and information on an evaluation score for the suitability of the established surgical plan determined based on pre-stored surgical data.

[0079] Additionally, a step (S8) of acquiring the user's CT image information and automatically generating 3D image information by reconstructing the CT image information in 3D is performed.

[0080] Here, the PSI size related to the CT image information can be changed using the cutting level on the surgical plan, and the 3D image information can be automatically generated by performing a 2D Boolean operation on the image information with the changed PSI size to fit the user's skeleton.

[0081] Thereafter, the first X-ray image information and template data are received, and STL information is generated using the automatically generated 3D image information, the first X-ray image information, and the template data together (S10), and the generated STL data is input to a 3D printer (S11).

[0082] Furthermore, a step (S12) of printing a surgical guide that simulates a surgical plan in advance based on STL data on a 3D printer can be performed.

[0083] The deep learning-based reverse shoulder arthroplasty preoperative planning system proposed by the present invention described above is an advanced technology solution that helps surgeons plan and optimize surgical procedures.

[0084] Reverse shoulder arthroplasty is a complex orthopedic surgery performed to treat shoulder conditions such as severe arthritis or rotator cuff tears. It involves reversing the normal anatomical structure of the shoulder joint to improve its function.

[0085] The present invention utilizes X-ray images to capture detailed anatomical information about a patient's shoulder joint, and these images are used as input data for a deep learning algorithm.

[0086] Deep learning algorithms, particularly convolutional neural networks (CNNs), excel at analyzing preprocessed medical images and extracting meaningful features from them. They have proven effective in various medical imaging tasks, extracting relevant features important for surgical planning.

[0087] These characteristics may include bone structure, joint angles, implant position, and muscle / tissue analysis.

[0088] Based on the extracted features, the system generates a detailed preoperative surgical plan, which may include determining the optimal implant type, size and position, cutting range and angle, predicting potential complications, predicting postoperative range of motion, and assessing the overall feasibility of the surgery.

[0089] The present invention proposes a comprehensive surgical planning system for reverse total shoulder arthroplasty, which automatically establishes a surgical plan before surgery for reverse total shoulder arthroplasty and automatically designs and produces a patient-specific surgical guide (Patient Specific Instrument) according to the established surgical plan for fast and accurate surgery.

[0090] Deep learning within the system can determine the cutting level of the acromion and even recommend products from specific companies to proceed with templating.

[0091] Furthermore, by using a Boolean algorithm to design a patient-specific surgical guide, a guide that can be used in surgery can be produced using 3D printing without separate design work.

[0092] Meanwhile, FIG. 3 illustrates an example of the UI configuration and function of a surgical planning system for retrograde total shoulder arthroplasty in relation to the present invention.

[0093] Figure 3 (a) shows examples of UI configurations in which image information (110) and surgical guides are designed (120) according to the final surgical plan.

[0094] Additionally, Fig. 3 (b) illustrates examples of steps S1 to S9 described above.

[0095] In relation to step S10 of receiving first X-ray image information and template data and generating STL information using automatically generated 3D image information, first X-ray image information, and template data together, steps S10a and S10b are shown separately.

[0096] Additionally, steps S11 and S12 are displayed over time.

[0097] In addition, FIG. 4 illustrates an example of a schematic diagram of a cutting level determination deep learning model in relation to the present invention.

[0098] Referring to (a) and (b) of FIG. 4, in relation to step S3, an example of the operation of the control unit (30) is shown for measuring the cutting level by 3D-composing a CT image on which surgery has been completed in advance, extracting a portion of an X-ray image of the same patient, labeling it, and then learning it.

[0099] Figure 4 (a) illustrates an example of the results of a cutting level determination deep learning model.

[0100] FIG. 5 illustrates an example of a templated deep learning model schematic diagram in relation to the present invention.

[0101] Referring to (a) and (b) of FIG. 5, in relation to step S3, an example of the operation of the control unit (30) that automatically determines the size and model for each shoulder joint image and templating the prepared component shape is illustrated.

[0102] Figure 5 (a) illustrates an example of the application result of a templating deep learning model.

[0103] FIG. 6 illustrates an example of a Boolean-based PSI automatic design algorithm in relation to the present invention.

[0104] Figure 6 illustrates the application of a Boolean-based PSI automatic design algorithm related to steps S8, S9, and S10.

[0105] Each step of (a) to (J) shows an example of a process of changing the previously stored basic PSI size using the cutting level determined from deep learning, performing 2D Boolean to fit the patient's skeleton, and reconstructing it in 3D.

[0106] When applying the system and method of the present invention described above, the present invention can accurately determine parts that are difficult for the surgeon to determine, such as the cutting level, by using deep learning, and can resolve the overall part of the surgical plan by recommending the most suitable artificial joint product for the patient based on images alone and even performing templating.

[0107] In addition, by providing designs for patient-tailored surgical guides, they can be produced using 3D printing without separate design work and applied directly to patient surgery, helping to ensure that the surgery can proceed according to the surgical plan.

[0108] In this case, the time required to establish a surgical plan and perform the surgical procedure can be drastically shortened, and the gap in the experience and skills of the surgeon can be reduced, allowing even less experienced surgeons to perform the surgery accurately, and the patient's pain due to the surgery can also be reduced.

[0109] Meanwhile, the effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects that are not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.

[0110] In addition, the system and its control method described above are not limited to the configuration and method of the embodiments described above, and the embodiments may be configured by selectively combining all or part of each embodiment so that various modifications can be made.

Claims

1. The first step is to obtain X-ray image information; A second step of performing pre-processing for analysis of a pre-designated area among the above X-ray image information; A third step of analyzing the above preprocessed X-ray image using a convolutional neural network (CNN) and extracting and learning feature information from the analysis results; A fourth step of repeating the first to third steps and generating template data for the pre-designated area based on the repeatedly learned feature information; Step 5 of acquiring the user's first X-ray image information; A sixth step of extracting first feature information related to the user from among the feature information using the first X-ray image information and the template data; and A deep learning-based surgical plan generation method comprising a seventh step of generating a surgical plan related to the user based on the first feature information.

2. In paragraph 1, The surgery related to the above user includes reverse total shoulder arthroplasty, The above-mentioned pre-designated area is a deep learning-based surgical plan generation method including the shoulder joint area.

3. In paragraph 2, The preprocessing in the second step is as follows: A deep learning-based surgical plan generation method that can analyze anatomical information about the shoulder joint area.

4. In paragraph 3, The above characteristic information is, A deep learning-based surgical plan generation method comprising at least one of anatomical structure information related to the user, bone structure information, angle information of a pre-specified joint, muscle and / or tissue analysis information, and information related to the size and position of an implant structure to be applied to the shoulder joint.

5. In paragraph 4, In the above generated surgical plan, A deep learning-based surgical plan generation method including at least one of information determining the type, size, and location of the implant to be applied to the shoulder joint, information on the cutting range and cutting angle for applying the reverse shoulder joint replacement, information on prediction of potential complications after the surgery, information on prediction of the user's range of motion related to the shoulder joint area after the surgery, and information on the suitability evaluation score of the established surgical plan determined based on pre-stored surgical data.

6. In paragraph 5, In the above first step, CT images of previously completed surgery are acquired together, In the third step above, Based on the above CT image, the cutting level to be applied to the completed surgery is measured, Some areas related to the completed surgery are extracted from the above X-ray image, A deep learning-based surgical plan generation method that labels the preprocessed X-ray image based on the above-described cutting level and the extracted partial area and then performs the above-described learning.

7. In paragraph 6, In the above 4th step, A deep learning-based surgical plan generation method in which a first shape is automatically determined based on the size of the selected region among a plurality of pre-stored component shapes based on an area selected by an external user from among the above X-ray image information, and the template data is generated based on the first shape.

8. In paragraph 7, After step 7 above, Step 8 of acquiring CT image information of the user; A ninth step of automatically generating three-dimensional image information by three-dimensionally reconstructing the above CT image information; and A 10th step of receiving the first X-ray image information and the template data, and generating STL information by using the automatically generated 3D image information, the first X-ray image information, and the template data together; Step 11 of inputting the above-generated STL data into a 3D printer; and A deep learning-based surgical plan generation method further comprising a 12th step of outputting a surgical guide that has applied the surgical plan in advance based on the STL data from the 3D printer.

9. In paragraph 8, In the above 9th step, Change the PSI size related to the CT image information using the cutting level in the above surgical plan, A deep learning-based surgical plan generation method that automatically generates three-dimensional image information by performing two-dimensional Boolean operations on image information with the PSI size changed to fit the user's skeleton.

10. An acquisition unit that acquires X-ray image information; A preprocessing unit that performs preprocessing for analysis of a pre-designated area of ​​the above X-ray image information; and A control unit that analyzes the above-mentioned preprocessed X-ray image using a convolutional neural network (CNN) and extracts and learns feature information from the analysis results; After repeating the operations of the above acquisition unit, preprocessing unit and control unit, the control unit generates template data for the pre-specified area based on the repeatedly learned feature information, If the above acquisition unit additionally acquires the user's first X-ray image information, A deep learning-based surgical plan generation device in which the control unit extracts first feature information related to the user from among the feature information using the first X-ray image information and the template data, and generates a surgical plan related to the user based on the first feature information.

Images