Bone tissue drilling state recognition method and system based on multi-sensor information fusion

Abstract

The invention discloses a bone tissue drilling state recognition method and system based on multi-sensor information fusion, belongs to the technical field of medical robot and operation state recognition, and realizes real-time recognition of different levels of bone tissues by synchronously collecting signals of a force sensor and an acceleration sensor. The method comprises the steps of signal preprocessing (filtering, normalization and time registration), feature extraction (time domain features such as RMS and slope, frequency domain features such as PSD and harmonic amplitude, and time-frequency domain features such as wavelet energy), feature fusion and dimension reduction (weighted fusion is combined with PCA, and 95% variance is reserved) and state recognition model construction (Multi-Kernel SVM, parameters and kernel weights are optimized through PSO). The system is composed of a signal acquisition module, a preprocessing module, a feature extraction and fusion module, a recognition model module and an output module, supports GUI real-time display and control instruction output, and realizes low-delay response. According to the invention, rapid and accurate identification of different levels of bone tissues can be realized, and drilling breakthrough and tissue damage risks are reduced.

Description

Technical Field

[0001] This invention belongs to the field of medical device and artificial intelligence technology, and relates to surgical robot-assisted systems and multimodal signal processing technology. Specifically, it relates to a method and system for identifying the state of bone tissue drilling based on multi-sensor information fusion. Background Technology

[0002] Bone drilling is a common procedure in orthopedic and neurosurgical procedures. Accurate identification of bone layers during drilling is crucial for avoiding drill bit breaches, reducing neurovascular damage, and improving surgical safety. Imaging techniques such as computed tomography (CT) and magnetic resonance imaging (MRI) are traditionally widely used for preoperative planning, bone density assessment, and navigation guidance. For example, CT images are used to reconstruct bone structure models to assist in drill path design and location, thereby improving surgical precision. However, CT is usually used as an auxiliary tool rather than a core means of real-time drilling status identification, still requiring manual operation and lacking dynamic feedback. Other existing methods mainly rely on the surgeon's clinical experience, manual tactile feedback, or single sensors (such as force sensors, vibration sensors, or sound sensors) for status monitoring. For example, changes in bone layers are judged by drilling force disturbances, constant-speed drilling, or the surgeon's tactile feedback. However, these methods suffer from signal instability, insufficient robustness, and poor real-time performance, leading to misjudgments or delayed responses in complex surgical environments.

[0003] In recent years, multi-sensor information fusion technologies (such as the combination of force and acceleration signals, or the fusion of acoustic and mechanical signals) have been introduced into the field of bone tissue state recognition, providing complementary information and improving recognition accuracy and robustness. Related research has been reported, for example, using acoustic signal-based cutting depth estimation models for surgical robot control, or monitoring tool wear and bone layer status through the fusion analysis of force and vibration signals. However, existing multi-sensor fusion methods still face several challenges: insufficient signal preprocessing, failing to effectively remove noise and artifacts; inadequate feature extraction, making it difficult for single time-domain or frequency-domain indicators to accurately distinguish transition layers and breakthrough states; and limited model generalization ability, with performance degrading under different bone densities, individual patient differences, or surgical scenarios. These problems limit the promotion and application of this technology in clinical practice, necessitating more advanced fusion strategies and optimized models to address them. Summary of the Invention

[0004] To address the aforementioned issues, this invention aims to provide a method and system for identifying the state of bone tissue drilling based on multi-sensor information fusion. This system can identify different layers of bone tissue in real time and accurately, including pre-contact, cortical bone, cancellous bone, transition layer, and breakthrough state.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: On the one hand, this invention proposes a method for identifying the state of bone tissue drilling based on multi-sensor information fusion, including the following process: S1: Multi-sensor synchronous signal acquisition: Force sensors are used to acquire axial thrust signals, and acceleration sensors are used to acquire vibration signals. A timestamp mechanism is used to achieve synchronous acquisition of multi-source signals to ensure data alignment. S2: Signal preprocessing: Filtering, normalizing and time-registering the acquired signals to remove noise, process bone density differences and ensure time axis alignment of force / acceleration signals; S3: Feature extraction: Extract time-domain, frequency-domain and time-frequency-domain features from the preprocessed signal to form a feature vector, which is used to characterize the changes in bone tissue state during drilling. S4: Feature Fusion and Dimensionality Reduction: The extracted features are weighted and fused, and redundancy is reduced through a dimensionality reduction algorithm; S5: Constructing a state recognition model: A Multi-Kernel SVM model is used to identify the state of bone tissue. The kernel function combination includes RBF kernel and linear kernel. The model parameters and kernel weights are optimized by PSO to achieve the classification of pre-contact, cortical bone, cancellous bone, transition layer and breakthrough state. S6: System Output: Based on the recognition results, the status of the bone drill is displayed in real time through the GUI, and control commands are output to the surgical robot or auxiliary device.

[0006] Furthermore, in step S1, the sampling frequency of the force sensor is 100Hz, and the sampling frequency of the acceleration sensor is 1000Hz.

[0007] Furthermore, step S2 specifically includes: Filtering: The force signal is filtered using a Butterworth second-order low-pass filter to remove high-frequency noise. The low-pass cutoff of the force signal is 10Hz. The acceleration signal is filtered using a 20-500Hz band-pass filter to focus on the vibration characteristics of the bone drill. Normalization: The Z-score method is used to handle amplitude shifts caused by differences in bone density, and to standardize the differences in amplitude between different bone densities or sensors. Time registration: The force signal is upsampled to 1000Hz through linear interpolation and aligned with the acceleration signal to achieve synchronous fusion of multiple sensors.

[0008] Furthermore, in step S3, features are extracted within each sliding window of 200ms and a step size of 100ms. The time-domain features include RMS, slope, and peak value, which are used for energy and rate of change analysis. The frequency-domain features include power spectral density and harmonic amplitude, which are used to analyze cortical bone energy distribution and resonance capture. The time-frequency domain features include wavelet energy, which are used to capture transition layers and detect breakthrough state abrupt changes. These features together constitute the feature vector for each time window, which is used to characterize the changes in bone tissue state during drilling.

[0009] Furthermore, in step S4, the feature fusion adopts a weighted mechanism to integrate the force signal features and acceleration signal features to enhance the complementarity of multi-source information; then, through principal component analysis, dimensionality reduction is performed to retain the variance of the main features and reduce the feature vector dimension to a preset dimension, thereby reducing redundancy and improving model training efficiency and real-time recognition capability.

[0010] Furthermore, in step S5, a Multi-Kernel SVM model is constructed, with kernel function combinations including RBF kernel and linear kernel; Particle Swarm Optimization (PSO) is used to automatically adjust SVM parameters C, gamma, and kernel weights to maximize classification accuracy; the model supports online or offline training, achieving high-precision classification of four types of bone tissue states, adapting to different bone densities and surgical scenarios.

[0011] Furthermore, in step S6, the system output is updated and displayed via a GUI every 0.1 seconds, and control commands are output via a serial port or TCP / IP interface, which can be linked with the surgical robot or a manual auxiliary drilling device to achieve low-latency response.

[0012] On the other hand, the present invention also proposes a bone tissue drilling state recognition system based on multi-sensor information fusion, for implementing the method described above, including: (1) Multi-sensor module: including single-axis force sensor and single-axis acceleration sensor; single-axis force sensor is used to acquire axial push-pull force signal with high precision, and single-axis acceleration sensor is used to acquire vibration signal during drilling process; (2) Signal acquisition module: connects force sensor and acceleration sensor, is equipped with processor and memory, and is equipped with data acquisition card. It supports multi-channel analog-to-digital conversion, and uses timestamp mechanism to align multi-source signals and supports multi-source synchronous acquisition; (3) Preprocessing module: Implements filtering, normalization and time registration; (4) Feature extraction and fusion module: Extract time-domain, frequency-domain, and time-frequency-domain features, and perform weighted fusion and PCA dimensionality reduction; (5) Identification model module: Multi-Kernel SVM and PSO optimization are used to classify bone tissue status; (6) Output module: The GUI displays the status and control command output in real time, supports the integration of surgical robots, and achieves low latency response.

[0013] Compared with the prior art, the technical solution of the present invention has the following beneficial effects: (1) Realize real-time, high-precision bone tissue status recognition, especially transitional layers and breakthrough states; (2) Improve surgical safety and reduce the risk of bone drill penetration and neurovascular injury; (3) It adapts to different bone densities and surgical scenarios, and has good generalization ability and robustness. Attached Figure Description

[0014] Figure 1 This is a process flow diagram of an embodiment of the present invention. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be described in detail below. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other implementation methods obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0016] like Figure 1 As shown, embodiments of the present invention detail the implementation details of a method and system for identifying the state of bone tissue drilling based on multi-sensor information fusion, including hardware components, software algorithms, data processing procedures, model training and optimization, system integration, and experimental verification. Each embodiment aims to ensure the repeatability and practicality of the invention. In real-world applications, both the data acquisition unit and the host computer utilize general-purpose computers to leverage their high-performance CPUs, memory, and software flexibility.

[0017] Example 1: System Overall Architecture, Hardware Design and Signal Preprocessing The system mainly consists of a multi-sensor module and a host computer processing unit. The host computer processing unit integrates a data acquisition module, a signal processing module, a status recognition module, and a user interface, and adopts a modular design to facilitate system expansion and maintenance.

[0018] The multi-sensor module includes a single-axis force sensor and a single-axis acceleration sensor. The force sensor is used to acquire axial push-pull force signals with high precision, while the acceleration sensor is used to acquire vibration signals during drilling, with a response frequency on the order of kilohertz. The sensors are connected to the data acquisition unit via USB or analog output. The force sensor can be fixed to the axial position of the bone drill, while the acceleration sensor can be mounted on the drill bit housing to avoid direct contact with bone tissue.

[0019] The data acquisition unit is a general-purpose computer equipped with a processor and memory, and a data acquisition card, supporting multi-channel analog-to-digital conversion. Force signals can be acquired at a frequency of 100Hz, and acceleration signals at a frequency of 1000Hz to capture different dynamic characteristics. To ensure synchronization, a timestamp mechanism is used for multi-source signal alignment. If a synchronization deviation exceeds a set threshold, abnormal data can be discarded and logged.

[0020] The host computer processing unit can share the same computer with the data acquisition unit to achieve an integrated mode, or it can be connected to another computer via a network to achieve distributed processing. The host computer can be configured with a graphics processing unit (GPU) to support real-time signal processing and artificial intelligence model inference. Its software environment can run data processing and machine learning frameworks to complete signal preprocessing, feature extraction, and model training.

[0021] In integrated mode, the same computer can execute data acquisition and signal processing tasks separately through multi-threading or parallel mechanisms, thereby avoiding resource conflicts and improving real-time performance.

[0022] During installation, this system only requires connecting the data acquisition card to the host computer via a standard interface and connecting the sensor cable to the corresponding channel to complete the hardware installation. It is suitable for operating room and laboratory environments. The software component includes the acquisition driver and signal processing modules, and is ready to run after installation.

[0023] When in use, users start the acquisition through the graphical user interface. The system automatically completes sensor initialization and zero-point calibration. The acquired data can be saved in real time for subsequent analysis, and it also supports real-time display of waveforms and processing results.

[0024] The signal processing module is used to preprocess the acquired raw signal, including steps such as filtering, normalization, and time registration.

[0025] Filtering section: The force signal uses a Butterworth second-order low-pass filter with a preferred low cutoff frequency to remove high-frequency noise.

[0026] The acceleration signal is filtered using a bandpass filter of an appropriate frequency band to focus on the vibration characteristics of drilling.

[0027] Normalization was performed using the Z-score method: x_norm = (x - μ) / σ, where μ and σ are the mean and standard deviation within the window. This method eliminates the influence of different bone densities on signal amplitude; for example, low bone density in elderly patients leads to weaker force signals.

[0028] Time registration upsamples the force signal to the acceleration sampling rate through linear interpolation, aligning it with the acceleration signal.

[0029] Example 2: Feature Extraction, Fusion, Dimensionality Reduction, and State Recognition Model A sliding window mechanism is employed in the time dimension to segment the acquired signals, with a window length of 200ms and a step size of 100ms. This ensures that the complete mechanical change cycle is captured in each window, while balancing real-time performance and data integrity. Time-domain features include root mean square (RMS, used for energy averaging), slope (reflecting signal change trends), and peak value (capturing transient impacts). Frequency-domain features include power spectral density (PSD) and harmonic amplitude values ​​to distinguish the energy distribution differences between cortical and cancellous bone. Wavelet transform is used in the time-frequency domain to calculate energy at different scales, capturing abrupt changes in transition layers and breakthrough states. This combination yields approximately 30-dimensional multi-source feature vectors within a single time window, providing rich information for subsequent identification.

[0030] In the feature fusion stage, a weighted mechanism is used to linearly combine the feature vectors of the force signal and the acceleration signal. The force signal provides a stable low-frequency trend, while the acceleration signal is sensitive to high-frequency dynamic changes; the two complement each other. The fusion weights can be dynamically adjusted according to the signal-to-noise ratio of the real-time signal, thereby improving the system's robustness in complex environments.

[0031] Principal component analysis (PCA) was used for dimensionality reduction, retaining over 95% of the cumulative variance contribution. After dimensionality reduction, the feature dimension was reduced from approximately 30 to about 10, which preserved the main information, reduced redundancy, and improved the model's efficiency and generalization ability in real-time environments.

[0032] In the design of the state recognition model, a multi-kernel support vector machine (Multi-Kernel SVM) is adopted. This model combines radial basis function kernels and linear kernels, possessing the ability to handle complex nonlinear features while maintaining a certain level of generalization performance. To optimize model parameters and kernel weights, a particle swarm optimization (PSO) algorithm is introduced for global search and adaptive adjustment. This method can theoretically achieve better classification performance, providing support for bone layer state recognition in complex environments.

[0033] The classification labels cover four typical drilling states: (1) pre-contact stage, with low force signal and insignificant vibration; (2) cortical bone stage, with relatively large thrust, manifested as strong axial thrust, but relatively small vibration amplitude and less pronounced fluctuation than cancellous bone stage; (3) cancellous bone stage, with relatively small thrust, large vibration amplitude and fluctuation, manifested as strong vibration response and significant fluctuation; (4) transition and breakthrough stage, manifested as a sudden drop in force signal and transient peak vibration. Especially in the breakthrough state, the fusion feature can accurately capture transient impacts and reduce the risk of missed detection.

[0034] By using the above-mentioned feature extraction, fusion, and modeling methods, real-time identification of different states during bone drilling can be achieved.

[0035] In summary, this invention addresses the problems of insufficient signal preprocessing, inadequate feature extraction, and limited model generalization ability in existing technologies by proposing a systematic improvement scheme: (1) Through multi-level filtering, Z-score normalization and timestamp synchronization mechanism, high-quality preprocessing and accurate alignment of multi-source signals are achieved; (2) Construct time-domain, frequency-domain and time-frequency domain joint feature models, extract multi-dimensional features such as RMS, harmonic amplitude and wavelet energy, and improve the recognition accuracy of transition layer and breakthrough state; (3) A multi-core support vector machine (combination of RBF kernel and linear kernel) is adopted and combined with particle swarm optimization algorithm to adaptively adjust parameters and kernel weights to enhance the classification accuracy and generalization ability of the model.

[0036] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A bone tissue drilling state recognition method based on multi-sensor information fusion, characterized by, The method comprises the following steps: S1: Multi-sensor synchronous signal acquisition: using a force sensor to collect axial thrust signals, and using an acceleration sensor to collect vibration signals, and realizing multi-source signal synchronous acquisition through a time stamp mechanism to ensure data alignment; S2: Signal preprocessing: filtering, normalizing and time registration of the collected signals to remove noise, process bone density differences and ensure alignment of the force / acceleration signal time axis; S3: Feature extraction: extracting time domain, frequency domain and time-frequency domain features from the preprocessed signals to form a feature vector for representing changes in the bone tissue state during drilling; S4: Feature fusion and dimension reduction: weighting fusion of the extracted features and reduction of redundancy through a dimension reduction algorithm; S5: Construction of a state recognition model: using a Multi-Kernel SVM model for bone tissue state recognition, the kernel function combination including an RBF kernel and a linear kernel, the model parameters and kernel weights being optimized through PSO to realize classification of pre-contact, cortical bone, cancellous bone, transition layer and breakthrough state; S6: System output: based on the recognition result, displaying the bone drill state in real time through a GUI and outputting control instructions to a surgical robot or an auxiliary device.

2. The bone drilling state recognition method based on multi-sensor information fusion according to claim 1, characterized in that, In the step S1, the sampling frequency of the force sensor is 100 Hz, and the sampling frequency of the acceleration sensor is 1000 Hz.

3. The bone drilling state recognition method based on multi-sensor information fusion according to claim 1, characterized in that, The step S2 is specifically: Filtering: Butterworth 2-order low-pass filter is used for the force signal to remove high-frequency noise, the force signal low-pass cutoff is 10 Hz, and a 20-500 Hz band-pass filter is used for the acceleration signal to focus on the bone drill vibration characteristics; Normalization: Z-score method is used to process the amplitude deviation caused by bone density differences to standardize the amplitude differences of different bone densities or sensors; Time registration: the force signal is up-sampled to 1000 Hz through linear interpolation to align with the acceleration signal, and multi-sensor synchronous fusion is realized.

4. The bone drilling state recognition method based on multi-sensor information fusion according to claim 1, characterized in that, In the step S3, features are extracted in each sliding window of 200 ms and step length of 100 ms, the time domain features include RMS, slope and peak value for energy and change rate analysis, the frequency domain features include power spectral density and harmonic amplitude for analyzing cortical bone energy distribution and resonance capture, and the time-frequency domain features include wavelet energy for capturing transition layer and breakthrough state mutation detection; The features collectively form a feature vector for each time window for representing changes in the bone tissue state during drilling.

5. The bone drilling state recognition method based on multi-sensor information fusion according to claim 1, characterized in that, In the step S4, the feature fusion adopts a weighting mechanism to integrate the force signal features and the acceleration signal features to enhance the complementarity of multi-source information, and then the principal component analysis is used for dimension reduction to retain the main feature variance and reduce the feature vector dimension to a preset dimension, thereby reducing redundancy and improving model training efficiency and real-time recognition capability.

6. The bone drilling state recognition method based on multi-sensor information fusion according to claim 1, characterized in that, In step S5, a Multi-Kernel SVM model is constructed, the kernel function combination includes an RBF kernel and a linear kernel; particle swarm optimization (PSO) is used to automatically adjust the SVM parameters C, gamma and kernel weight, and to maximize the classification accuracy; the model supports online or offline training, realizes high-precision classification of the four types of bone tissue states, and is suitable for different bone densities and surgical scenarios.

7. The bone drilling state recognition method based on multi-sensor information fusion according to claim 1, characterized in that, In step S6, the system output is updated and displayed every 0.1 s using a GUI, and control instructions are output through a serial port or a TCP / IP interface, so that a surgical robot or a manual auxiliary drilling device can be linked to realize low-delay response.

8. A bone drilling state recognition system based on multi-sensor information fusion, for implementing the method of any one of claims 1-7, characterized in that, It comprises: (1) a multi-sensor module, including a single-axis force sensor and a single-axis acceleration sensor; the single-axis force sensor is used to collect axial push-pull force signals with high precision, and the single-axis acceleration sensor is used to collect vibration signals in the drilling process; (2) a signal acquisition module, which is connected to the force sensor and the acceleration sensor, is configured with a processor and a memory, is equipped with a data acquisition card, supports multi-channel analog-digital conversion, and uses a timestamp mechanism to align multi-source signals and support multi-source synchronous acquisition; (3) a preprocessing module, which realizes filtering, normalization and time registration; (4) a feature extraction and fusion module, which extracts time domain, frequency domain and time-frequency domain features, and performs weighted fusion and PCA dimension reduction; (5) a recognition model module, which uses Multi-Kernel SVM and PSO optimization to realize bone tissue state classification; (6) an output module, which displays the state and control instruction output in real time using a GUI, supports surgical robot integration, and realizes low-delay response.

Images