A method for identifying bone layers in a hard tissue surgery
By combining finite element simulation models and deep learning models, the real-time and accuracy issues of bone layer identification in hard bone tissue surgery were solved, enabling real-time identification and precise positioning of bone layers in hard bone tissue surgery, and reducing the number of sensors required.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING INST OF TECH
- Filing Date
- 2023-08-17
- Publication Date
- 2026-07-07
AI Technical Summary
Existing technologies struggle to accurately locate the inner cortical bone in real time during surgery on hard bone tissues, and the complex sensor fusion methods result in low recognition accuracy and are unsuitable for surgical tools.
Data annotation is guided by a finite element simulation model, combined with a deep learning model and a Gaussian mixture model. Force sensors are used to collect and implement interactive force signals. A Gaussian mixture model is established and trained using a convolutional neural network model to describe the distribution. The distribution of bone layer features is analyzed using real-time force signals. The finite element simulation model guides data annotation, and the bone layer is identified by combining the data with a deep learning model.
It enables real-time identification of bone layers during hard bone tissue surgery, reduces the number of sensors used, improves identification accuracy, and is applicable to various hard bone tissue surgical environments.
Smart Images
Figure CN117297769B_ABST
Abstract
Description
Technical Field
[0001] This invention provides a method for identifying bone layers during surgery on hard bone tissue, belonging to the field of image processing technology. Background Technology
[0002] Traditional bone layer identification utilizes imaging methods such as X-rays, CT scans, and MRI to provide image information of skeletal structures. However, accurate localization of the inner cortical bone presents challenges. The inner cortical bone is close to nerves and is often obscured by soft tissues, fat, and blood vessels, making accurate identification during surgery difficult. Furthermore, image processing and computer vision technologies are used to automatically or semi-automatically extract the location information of the inner cortical bone from medical images. Commonly used methods include edge detection, thresholding, region growing, and morphological manipulation. These methods can identify and segment bone structures based on their characteristics, density, and morphology, providing more accurate surgical navigation and localization.
[0003] Image-based bone layer identification methods can only be completed preoperatively and cannot be performed in real time based on signals. Furthermore, image-based bone layer identification requires doctors to annotate each layer individually, which increases the workload significantly.
[0004] One existing technology (Hand-held bone cutting tool with autonomous penetration detection for spinal surgery, IEEE) proposes a handheld bone cutting tool system that can detect the penetration force of a workpiece. The system learns the cutting and motion states from a surgeon's demonstration and autonomously detects workpiece penetration, immediately stopping the cutting tool's drive before complete penetration. The proposed penetration detection scheme does not require knowledge of the workpiece's shape and position, thus eliminating the need for expensive systems such as robotic arms and position sensor systems. Furthermore, the proposed scheme can be easily applied to cutting tools of various shapes. The developed system was evaluated experimentally. Results show that the developed system performs satisfactorily in both electric and handheld settings. This technology has the following drawbacks:
[0005] 1. This method is only applicable to handheld tools and can only differentiate between the doctor's operations, but cannot identify bone layers.
[0006] 2. Surgical tools need to be controllable in real time, so they need to be customized, which cannot be met by the surgical tools used in hospitals.
[0007] CN202210471044.7 provides a method for identifying chatter types in robot milling based on power spectral entropy difference. The method includes the following steps: during robot milling, acquiring the original vibration signal at the robot's end effector; determining the optimal number of mode decompositions, and decomposing the original vibration signal into multiple sub-signals based on the optimal number of mode decompositions; denoting the sub-signal with a center frequency close to the natural frequency of the tool-spindle system as signal B1, and the remaining sub-signals below the natural frequency as A1; filtering out the spindle rotation frequency and harmonic components in signals A1 and B1 to obtain signals A2 and B2; calculating the power spectral entropy of each signal to obtain the power spectral entropy difference; determining the optimal classification threshold for the power spectral entropy difference, and performing chatter type identification. This invention comprehensively considers regenerative chatter caused by the flexibility of the tool-spindle structure and modal coupling chatter caused by insufficient robot structural stiffness, thus achieving chatter type identification in robot milling. However, this technology has the following drawbacks:
[0008] 1. Vibration signals are used for identification, but vibration signals are greatly affected by the environment. In addition, due to the uneven material composition of hard bone materials, it is difficult to obtain effective vibration characteristics.
[0009] 2. Vibration signal mode decomposition easily loses important features.
[0010] 3. This method is suitable for metal processing, but the natural frequency of hard bone tissue is low, so it cannot be applied to hard bone tissue.
[0011] Existing technology (Force perception and bone recognition of vertebral laminamilling by robot-assisted ultrasonic bone scalpel based on backpropagation neural network, IEEE) proposes a robotic system that uses an ultrasonic bone scalpel to measure the milling force of the vertebral laminae, achieving a safe milling strategy. The developed bone recognition model based on a backpropagation neural network is applicable to robot-assisted vertebral laminae milling using milling layering and recognition algorithms. This model uses characteristic milling force, milling speed, milling depth, and ultrasonic scalpel power as inputs to determine whether milling has reached the intracortical bone, thus identifying and judging the bone layer. In vivo animal validation experiments show that the model can accurately determine the safe milling endpoint. In summary, this recognition model can significantly improve the safety and reliability of robot-assisted laminae resection, showing significant translational potential. However, this technology has the following drawbacks:
[0012] 1. Backpropagation (BP) neural networks typically require a large amount of labeled data for training, especially in complex tasks and multi-class classification problems. If the labeled data is limited, the network may suffer from underfitting and fail to fully utilize the information in the data.
[0013] 2. If the input data varies greatly in range or is unevenly distributed, the training and performance of the network may be affected. Normalization or standardization of the input data is usually necessary to reduce this impact.
[0014] Existing technology (State recognition of decompressive laminectomy with multiple information in robot-assisted surgery, IEEE) proposes a state recognition system for robot-assisted remote surgery. By combining learning methods with traditional methods, the slave robot can think about the current operational state like a surgeon and provide more information and decision suggestions to the master surgeon, which helps the surgeon work more safely in remote surgery. For fenestration, we propose an image-based state recognition method consisting of a U-Net derived network, grayscale redistribution, and dynamic receptive field, which helps control the grinding process to prevent the grinding head from penetrating the inner edge of the blade and damaging the spinal nerve. For internal fixation, we propose an audio and force-based state recognition method consisting of signal feature extraction, LSTM-based prediction, and information fusion, which assists in monitoring the drilling process to prevent the drill head from penetrating the outer edge of the vertebral pedicle and damaging the spinal nerve. This technology has the following drawbacks:
[0015] 1. Using sound and force signals requires more sensors.
[0016] 2. LSTM-based predictions have low accuracy and limited prediction range, making them unsuitable for bone layer identification.
[0017] Existing technology (Tactile perception for surgical status recognition in robot-assisted laminectomy, IEEE) is based on human tactile perception mechanisms and proposes a surgical status perception method utilizing accelerometers and force sensors mounted on a robot. A Sinc convolutional layer is introduced to process high-frequency vibration signals, and a one-dimensional convolutional network is used to process the obtained features together with the average force signal. This method classifies surgical status into one category and outputs the probability of burr blockage. Experiments on animal skeletons validate the effectiveness of the proposed model. Furthermore, it is demonstrated that the fusion of the two tactile signals can significantly improve the status recognition accuracy under varying milling parameters. This technique has the following drawbacks:
[0018] 1. All data annotations are based on subjective judgment and may differ from the actual situation.
[0019] 2. Accelerometers and force sensors increase the computational load on the network.
[0020] 3. The weights of the two signals are not differentiated because the accelerometer and the force sensor have different accuracies. Summary of the Invention
[0021] This invention provides a method for identifying bone layers in hard bone tissue surgery, mainly to solve the problem of identifying cortical bone and cancellous bone during hard bone tissue surgery.
[0022] The specific technical solution provided by this invention is as follows:
[0023] 1. A method for identifying bone layers during surgery on hard bone tissue, characterized by comprising the following steps:
[0024] Step 1: Establish a finite element simulation model of surgical procedures on bone tissue; perform geometric modeling of bone tissue, discretize it into a finite element mesh, define the material properties of bone tissue, and introduce constraints of the operation process into the finite element model.
[0025] Step 2: During the simulation, different surgical operation scenarios are considered, and the corresponding changes in interaction force are obtained based on different parameters;
[0026] Step 3: The bone tissue surgical robot performs grinding, drilling, and cutting operations according to the preoperative plan; during the operation, real-time interaction force signals between the surgical tools and the bone tissue are collected;
[0027] Step 4: To adapt to different bone materials and surgical parameters, multiple different types of bone layer features are selected for the interaction force signal; subsequently, the force interaction curves obtained through the finite element simulation model are used to guide the annotation of the outer cortical bone, cancellous bone and inner cortical bone under different bone layer features.
[0028] Step 5: Establish a Gaussian mixture model to describe the distribution of different bone layer features;
[0029] Step 6: Train the network model to accurately identify different bone layers. In practical applications, the trained network model is used to identify bone layers based on real-time force signals.
[0030] Furthermore, the material properties of the bone tissue in step 1 include elastic modulus, shear modulus, and Poisson ratio.
[0031] Step 3 involves installing a force sensor at the end of the robotic arm, while the surgical tool is mounted on the movable end face of the force sensor. During the surgical procedure, the force sensor detects changes in the force signal, and the interaction force signal between the surgical tool and the bone tissue is collected by acquiring the signal sensed by the force sensor.
[0032] The bone layer characteristics described in step 4 are based on the amplitude, frequency, time domain, and frequency domain characteristics of the force signal.
[0033] In step 5, the bone layer features are set into three categories: peak force, root mean square force, and spectral features. Then, the weights and probability density functions of different features are estimated according to the feature distribution. The weight information of different bone layer features is obtained by clustering and distribution modeling the feature vectors. Based on the bone layer features established by the labeled dataset and Gaussian mixture model, a neural network model is designed and trained.
[0034] The neural network model is a deep learning model, including convolutional neural networks or recurrent neural networks, used to learn the correlation between force signals and bone layer categories.
[0035] In step 6, the trained network model is used to identify bone layers from real-time force signals; by inputting the real-time force signals into the network model, the model will output the corresponding bone layer category.
[0036] The main purpose of this invention is to propose a bone layer identification method for robot-assisted surgery of hard bone tissue, solving the problem of tremors in various hard bone tissue surgeries; it has the following technical effects:
[0037] 1. Use finite element simulation models to guide data annotation, rather than relying on subjective judgment;
[0038] 2. The finite element model is applicable to various types of bone tissue and can be adjusted according to the actual surgical situation.
[0039] 3. By using only force sensors, rather than fusion of multiple sensors, the signal acquisition and processing process is simplified.
[0040] 4. Establishing multiple types of features can improve training accuracy.
[0041] 5. The feature weights of each set of force signal data are adjusted using a Gaussian mixture model to maximize the effectiveness of the signal.
[0042] 6. The trained model can be used for real-time bone layer identification and for safety decisions in surgical robots. Attached Figure Description
[0043] Figure 1 This is a flowchart of the present invention. Detailed Implementation
[0044] This invention provides a method for identifying bone layers during surgery on hard bone tissue. The overall process is as follows: Figure 1 The specific steps are as follows:
[0045] Step 1: Establish a finite element simulation model of surgical procedures on bone tissue. Geometrically model the bone tissue, discretize it into a finite element mesh, and define the material properties of the bone tissue, such as elastic modulus, shear modulus, and Poisson ratio. Introduce constraints for the surgical procedure into the finite element model.
[0046] Step 2: During the simulation, different surgical scenarios can be considered, such as bone cutting, nail insertion, and screw tightening. Surgical tools and procedures should also be configured with different parameters based on the actual situation, such as the diameter and cutting edge of the surgical tools, feed speed, drilling speed, and feed angle. Based on these parameters, the corresponding changes in interaction force can be derived.
[0047] Step 3: Surgery on hard bone tissue requires the robot to perform grinding, drilling, and cutting operations according to the pre-operative plan. During these operations, real-time force signals between the surgical tools and the hard bone tissue are collected. Specifically, a force sensor is installed at the end of the robotic arm, and the surgical tools are mounted on the movable end face of the force sensor. During the surgery, the force sensor detects changes in the force signal, and by acquiring the signal sensed by the force sensor, the force signals between the surgical tools and the hard bone tissue can be collected.
[0048] Step 4: To accommodate different bone materials and surgical parameters, multiple different types of bone layer features can be selected for the interaction force signal. These features can be based on the amplitude, frequency, time-domain, and frequency-domain characteristics of the force signal. For example, peak force, root mean square force, frequency spectrum characteristics, and time-domain statistical characteristics can be selected as bone layer features. Subsequently, the force interaction curves obtained through the finite element simulation model are used to guide the annotation of the outer cortical bone, cancellous bone, and inner cortical bone under different bone layer features. The purpose of the annotation is to determine the corresponding changes of different bone layers under different features.
[0049] Step 5: Establish a Gaussian Mixture Model (GMM) to describe the distribution of different bone layer features. Bone layer features are categorized into three types: peak force, root mean square force, and spectral features. The weights and probability density functions of different features are estimated based on their distribution. Weight information for different bone layer features is obtained by clustering and distribution modeling the feature vectors. Based on the labeled dataset and the bone layer features established using the Gaussian Mixture Model, an appropriate neural network model is designed and trained. This can be a deep learning model, such as a convolutional neural network (CNN) or a recurrent neural network (RNN), used to learn the correlation between force signals and bone layer categories.
[0050] Step 6: Train the network model to accurately identify different bone layers. In practical applications, the trained network model is used to identify bone layers based on real-time force signals. By inputting the real-time force signal into the network model, the model outputs the corresponding bone layer category. This enables real-time bone layer identification and allows for corresponding decisions or control as needed.
Claims
1. A bone layer identification system for hard bone tissue surgery, characterized in that, The bone layer identification method includes the following steps: Step 1: Establish a finite element simulation model of surgical procedures on bone tissue; perform geometric modeling of bone tissue, discretize it into a finite element mesh, define the material properties of bone tissue, and introduce constraints of the operation process into the finite element model. Step 2: During the simulation, different surgical operation scenarios are considered, and the corresponding changes in interaction force are obtained based on different parameters; Step 3: The bone tissue surgical robot performs grinding, drilling, and cutting operations according to the preoperative plan; during the operation, real-time interaction force signals between the surgical tools and the bone tissue are collected; Step 4: To adapt to different bone materials and surgical parameters, multiple different types of bone layer features are selected for the interaction force signal; subsequently, the force interaction curves obtained through the finite element simulation model are used to guide the annotation of the outer cortical bone, cancellous bone and inner cortical bone under different bone layer features. Step 5: Establish a Gaussian mixture model to describe the distribution of different bone layer features; Bone layer features are set into three categories: peak force, root mean square force, and spectral features. The weights and probability density functions of different features are estimated based on the feature distribution. The weight information of different bone layer features is obtained by clustering and distribution modeling the feature vectors. Based on the bone layer features established by the labeled dataset and Gaussian mixture model, a neural network model is designed and trained. Step 6: Train the network model to accurately identify different bone layers. In practical applications, use the trained network model to identify bone layers based on real-time force signals.
2. The bone layer identification system in hard bone tissue surgery according to claim 1, characterized in that, The material properties of the bone tissue in step 1 include elastic modulus, shear modulus, and Poisson ratio.
3. The bone layer identification system in hard bone tissue surgery according to claim 1, characterized in that, Step 3 involves installing a force sensor at the end of the robotic arm, while the surgical tool is mounted on the movable end face of the force sensor. During the surgical procedure, the force sensor detects changes in the force signal, and the interaction force signal between the surgical tool and the bone tissue is collected by acquiring the signal sensed by the force sensor.
4. The bone layer identification system in hard bone tissue surgery according to claim 1, characterized in that, The bone layer characteristics described in step 4 are based on the amplitude, frequency, time domain, and frequency domain characteristics of the force signal.
5. The bone layer identification system in hard bone tissue surgery according to claim 1, characterized in that, The neural network model is a deep learning model, including convolutional neural networks or recurrent neural networks, used to learn the correlation between force signals and bone layer categories.
6. The bone layer identification system in hard bone tissue surgery according to claim 1, characterized in that, In step 6, the trained network model is used to identify bone layers from real-time force signals. By inputting the real-time force signals into the network model, the model will output the corresponding bone layer category.