Deep learning-based operation power tool doctor operation intention identification method

Abstract

The invention discloses a deep learning-based operation power tool doctor operation intention identification method, which is characterized in that balance between a filtering effect and a response speed is improved based on an SSNR adaptive filtering switching mechanism, different noise spectrums are automatically adapted through online optimization of IIR filter coefficients, and manual readjustment is not needed; meanwhile, by introducing a Top-k multi-scale period extraction strategy, the sensing ability of the model to weak period signals is enhanced. The recognition method solves the problems that in the prior art, doctor operating force and tool counter-acting force are seriously coupled, high-strength vibration noise interference is remarkable when a power tool runs, and the real operation intention of a doctor cannot be accurately recognized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of surgical robots, and particularly relates to a surgical power tool doctor operation intention recognition method based on deep learning. BACKGROUND

[0002] Compared with traditional surgery, robot-assisted surgery can significantly improve the precision and stability of operation. In the process of doctor-robot interaction and cooperation, accurately capturing the operation intention of the doctor is crucial.

[0003] At present, human-computer interaction control is a commonly used control method in the field of surgical robots. A six-dimensional force / torque sensor is installed at the end of the mechanical arm to perceive external forces, and a surgical power tool is installed on the other side of the force sensor. The doctor directly holds the end of the surgical power tool for operation, and the mechanical arm assists in execution.

[0004] However, due to the tool structure and the requirements of the surgical scene, the doctor's operation point and the surgical power tool are located on the same side of the sensor, resulting in a high coupling of the operation force applied by the doctor and the reaction force of the surgical power tool when working at the sensor, which is difficult to separate. More importantly, the strong vibration generated when the surgical power tool is turned on can seriously interfere with the readings of the sensor, significantly reducing the perception accuracy of the doctor's operation force, and directly affecting the precision and safety of the surgical operation.

[0005] In terms of human-computer cooperation, there is little research on the influence of high-intensity external noise on the perception of the doctor's operation force. For example, the invention patent application with publication number CN118143939A provides a machine-human physical cooperation sharing variable admittance control method based on force perception. Although its control model can adjust in real time according to the operator's movement intention, its model assumes that the operator and the mechanical arm are located on opposite sides of the operating object, which does not match the operation mode in which the doctor and the power tool are located on the same side of the sensor in orthopedic surgery. And this method relies on a relatively stable force signal environment when interacting with objects, and does not effectively solve the problem of serious pollution of high-intensity vibration noise to force perception signals when the surgical power tool is turned on.

[0006] Therefore, the technical problems to be solved in the prior art are: 1. Precisely separate the doctor's applied force and the tool's reaction force in the coupled signals output by the six-dimensional force sensor; 2. For high-intensity vibration noise during the start and cutting process of the power tool, effectively suppress and extract features; 3. Combine the end-to-end deep neural network model of multi-scale frequency domain and time sequence feature extraction to automatically learn the time sequence, frequency spectrum and weak periodic information of the force signal, and identify the doctor's real operation intention in real time and robustly. SUMMARY

[0007] The application provides a surgical power tool doctor operation intention recognition method based on deep learning to solve the problems that in the prior art, the doctor operation force is seriously coupled with the tool reaction force, the high-intensity vibration and noise interference during the running of the power tool is significant, and the real operation intention of the doctor cannot be accurately recognized.

[0008] In order to achieve the above-mentioned purpose, the application adopts the following specific technical solutions:

[0009] The surgical power tool doctor operation intention recognition method based on deep learning comprises the following steps:

[0010] Step 1, training phase:

[0011] Load the data set, encapsulate the six-dimensional force / torque channel data in the CSV (Comma-Separated Values, character-separated value) file and the state label into training samples in a sliding window with a length of L, wherein each sample X L×6 and the label y

[0012] A multi-layer time sequence network structure model based on TimesNet is adopted, the input dimension is 6 channels, the feature dimension is d, there are E encoding layers, and the time domain and frequency domain features are extracted through multi-scale convolution in each encoding layer;

[0013] A Top-k multi-scale period extraction strategy is applied, multi-core convolution is performed after significant period components are selected, a cross-entropy loss is used for the convolution model and is improved in combination with label smoothing and FocalLoss, and the loss function is:

[0014]

[0015] Wherein, C is the number of categories, y c is a real label indication, is a model prediction probability, and alpha is a smoothing coefficient;

[0016] An Adam optimizer is used in the training process, the learning rate is eta, and the optimal model parameters are dynamically saved after each epoch according to the validation set accuracy;

[0017] Step 2, calibration phase:

[0018] Six-dimensional force data in N groups of static states are collected, and a reference vector is calculated to eliminate zero drift:

[0019]

[0020] Wherein, x (j) represents the jth six-dimensional reading vector, is a calibration reference vector, and N is the number of reference samples;

[0021] Zero-point drift correction is achieved by element-wise subtraction of the reference value:

[0022]

[0023] where x[n] is the n-th frame of original six-dimensional data, x corr [n] is the corrected data;

[0024] The corrected signal is adaptively filtered, and the short-time signal-to-noise ratio (SSNR) is used to switch between IIR (Infinite Impulse Response) filtering and sliding mean filtering;

[0025] IIR filtering is enabled when SSNR is below a threshold γ, otherwise sliding mean filtering with window length W is used;

[0026] IIR Butterworth filter coefficients are automatically redesigned every Δ frames according to the peak value of the signal power spectrum in the noise window;

[0027] Step three, recognition stage:

[0028] The force signal when the doctor holds the surgical tool to operate is stored in the time sequence buffer sequence, and the time sequence buffer with a length of L is constructed every time a frame of filtered data is received Input X ∈ R L×6 and perform forward inference, output class prediction

[0029] Further, in step one, when extracting time domain and frequency domain features, the time domain-frequency domain conversion is realized by real fast Fourier transform:

[0030] X f = rfft(X)

[0031] where X ∈ R L×6 is the input sequence with a length of L, and X f is the frequency domain representation.

[0032] Further, in step two, the formula for calculating the short-time signal-to-noise ratio SSNR is:

[0033]

[0034] Further, in step two, the difference equation of IIR Butterworth low-pass filtering is:

[0035]

[0036] where M is the filter order, b k , a kFilter coefficients.

[0037] Further, the calculation formula of the sliding mean filter is:

[0038]

[0039] Wherein, W is the number of sliding window samples;

[0040] Further, in step two, gamma and W can be adjusted online.

[0041] Further, after step three, it further includes:

[0042] Step four, evaluation optimization stage:

[0043] After receiving the user termination instruction, the running time T and the processing frame number M are counted, the actual sampling frequency f actual is calculated, and the original data, the filtering result and the prediction label are exported as CSV files according to the time stamp for subsequent analysis and visualization; otherwise, step three is continued.

[0044] Further, the calculation formula of the actual sampling frequency f actual is:

[0045] f actual =M / T.

[0046] Compared with the prior art, the technical scheme of the present application has the following beneficial effects:

[0047] 1. The adaptive filtering switching mechanism based on SSNR realizes the balance between accurate noise suppression and real-time response, and improves the balance between filtering effect and response speed.

[0048] By introducing the short-time signal-to-noise ratio (SSNR) index, the present application can dynamically switch between IIR low-pass filtering and sliding mean filtering when the signal environment changes. When the power tool is started and the vibration is intense, the system automatically adopts IIR filtering to strengthen the low-frequency component and suppress high-frequency noise; when the vibration is weakened and more details need to be preserved, it switches to sliding mean filtering to minimize the phase delay. This mechanism not only effectively suppresses the pollution of high-intensity vibration noise on the force sensing signal, but also controls the system processing delay within 50ms while ensuring the filtering effect, thereby improving the real-time and robustness of the doctor's operation intention recognition.

[0049] 2. Introducing the Top-k multi-scale period extraction strategy enhances the perception ability of weak periodic signals.

[0050] Aiming at the weak periodic motion law in the operation process of the doctor and the multi-frequency components caused by tool vibration, the Top-k frequency domain screening and Inception multi-scale convolution module are added in the TimesNet architecture. First, the k most significant periodic components in the frequency spectrum are quickly located through FFT (Fast Fourier Transform), and then parallel convolution is performed on different scales to capture periodic and sub-periodic features, thereby significantly improving the sensitivity and classification recognition ability of the model to weak periodic signals. This strategy enables the subtle vibration features generated when the power tool interacts with the tissue to be effectively separated and identified, further improving the accuracy of the doctor's intention classification and the anti-interference ability of the system.

[0051] 3. Online optimization of IIR filter coefficients, automatic adaptation to various noise spectra without manual retuning.

[0052] The online filter coefficient optimization module is added in the adaptive filtering framework, and the numerator and denominator coefficients of the Butterworth filter are automatically adjusted according to the real-time signal-to-noise ratio and tool running state, so that the filter performance matches the current noise spectrum characteristics. This method avoids the repeated manual debugging caused by tool type or cutting material differences in traditional methods, significantly improves the adaptability and maintenance efficiency of the system in different surgical scenarios, and ensures stable extraction of the operation force signal applied by the doctor when the noise environment changes rapidly. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1 The flowchart of the surgical power tool doctor operation intention recognition method of the present application;

[0054] Figure 2 The schematic diagram of human-machine cooperation principle.

[0055] Among them, 1-mechanical arm, 2-doctor holding area, 3-surgical power tool, 4-patient joint in operation area. DETAILED DESCRIPTION

[0056] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0057] As Figure 1 shown, the embodiment of the present application provides a surgical power tool doctor operation intention recognition method based on deep learning, which includes the following steps:

[0058] Step 1, training phase:

[0059] After starting the software, the dataset is first loaded. The six-dimensional force / torque channel data and state labels in each CSV file are encapsulated into training samples using a sliding window of length L, where each sample X∈R L×6 And the labels y∈0,…,C-1.

[0060] The model employs a multi-layer temporal network structure based on TimesNet, with a 6-channel input dimension and a feature dimension of d. It has E encoding layers, each extracting temporal and frequency domain features through multi-scale convolution. The time-domain to frequency-domain transformation is achieved using real-valued Fast Fourier Transform.

[0061] X f =rfft(X)

[0062] Where, X∈R L×6 Given an input sequence of length L, X f It is represented in the frequency domain.

[0063] In the Inception module, a Top-k multi-scale periodicity extraction strategy is applied. Significant periodic components are selected and then subjected to multi-kernel convolution, significantly enhancing the ability to perceive weak periodic signals. The model employs cross-entropy loss combined with label smoothing and FocalLoss improvements. The loss function is:

[0064]

[0065] Where C is the number of categories, y c This is an indication of the actual label. Here, α represents the predicted probability of the model, and α is the smoothing coefficient.

[0066] The Adam optimizer is used during training with a learning rate of η, and the optimal model parameters are dynamically saved after each epoch based on the accuracy on the validation set. Deep learning is achieved through training.

[0067] Step 2, Calibration Phase:

[0068] Collect N sets of six-dimensional force data under static conditions and calculate the reference vector to eliminate zero-point drift:

[0069]

[0070] Where, x (j) This represents the six-dimensional reading vector of the j-th group. The calibration baseline vector is N, where N is the number of baseline samples.

[0071] Zero-point drift correction is achieved by subtracting the reference value element by element:

[0072]

[0073] Where x[n] is the original six-dimensional data of the nth frame, x corr [n] represents the corrected data.

[0074] After correction, the signal enters the adaptive filtering module, and the system switches between IIR filtering and moving average filtering based on the short-time signal-to-noise ratio (SSNR).

[0075]

[0076] IIR filtering is enabled when the SSNR is below the threshold γ; otherwise, a moving average filter with a window length of W is used. γ and W can be adjusted online.

[0077] The difference equation for the IIR Butterworth low-pass filter is as follows:

[0078]

[0079] Where M is the filter order, b k a k These are the filter coefficients.

[0080] The Butterworth filter coefficients are automatically redesigned every Δ frames based on the peak value of the signal power spectrum in the noise window, adapting to different noise distributions without manual intervention.

[0081] The formula for calculating the moving average filter is as follows:

[0082]

[0083] Where W is the number of samples in the sliding window.

[0084] Step 3, Identification Stage:

[0085] In real-time operation, the surgeon holds the surgical instruments and performs the procedures, storing the force signals during the operation into a timing buffer sequence. The system maintains a timing buffer of length L, and constructs the input X∈R for each frame of filtered data received. L×6 It then performs forward inference and outputs a category prediction.

[0086] Step 4, Evaluation and Optimization Phase:

[0087] After receiving the user's termination command, the runtime T and the number of processed frames M are counted, and the actual sampling frequency f is calculated. actual The original data, filtering results, and predicted labels are exported as CSV files by timestamp for easy subsequent analysis and visualization.

[0088] Actual sampling frequency f actual The calculation formula is:

[0089] factual =M / T.

[0090] The above identification method is implemented using Python software, specifically PyCharm 2024.2.5 (Community Edition).

[0091] Figure 2 This is a schematic diagram of the human-machine collaboration principle when using the surgical power tool doctor's operation intention recognition method of the present invention. A surgical power tool 3 is installed at the end of the robotic arm 1. One end of the surgical power tool 3 is provided with a doctor's grip area 2, and the other end is aligned with the patient's joint 4 in the surgical area.

[0092] Obviously, those skilled in the art can make various modifications and variations to the embodiments of the present invention without departing from the spirit and scope of the invention. Therefore, if these modifications and variations fall within the scope of the claims of the present invention and their equivalents, the present invention also intends to include these modifications and variations.

Claims

1. A method for recognizing surgeon's operational intent using surgical dynamic tools based on deep learning, characterized in that, Includes the following steps: Step 1, Training Phase: Load the dataset and encapsulate the six-dimensional force / torque channel data and state labels from the CSV file into training samples using a sliding window of length L, where each sample X∈R. L×6 And labels y∈0,…,C-1; A multi-layer temporal network structure model based on TimesNet is adopted, with an input dimension of 6 channels and a feature dimension of d. There are a total of E coding layers. Within each coding layer, temporal and frequency domain features are extracted through multi-scale convolution. A Top-k multi-scale periodicity extraction strategy is applied, and after selecting significant periodic components, multi-kernel convolution is performed. The convolution model uses cross-entropy loss combined with label smoothing and FocalLoss improvements. The loss function is: Where C is the number of categories, y c This is an indication of the actual label. The model predicts the probability, and α is the smoothing coefficient; The Adam optimizer is used during training with a learning rate of η, and the optimal model parameters are dynamically saved after each epoch based on the accuracy on the validation set. Step 2, Calibration Phase: Collect N sets of six-dimensional force data under static conditions and calculate the reference vector to eliminate zero-point drift: Where, x (j) This represents the six-dimensional reading vector of the j-th group. The calibration baseline vector is N, where N is the number of baseline samples. Zero-point drift correction is achieved by subtracting the baseline value element by element. Where x[n] is the original six-dimensional data of the nth frame, x corr [n] represents the corrected data; The corrected signal is adaptively filtered, switching between IIR filtering and moving average filtering based on the short-time signal-to-noise ratio (SSNR). IIR filtering is enabled when the SSNR is below the threshold γ; otherwise, a moving average filter with a window length of W is used. The IIR Butterworth filter coefficients are automatically redesigned every Δ frames based on the peak value of the signal power spectrum in the noise window. Step 3, Identification Stage: The force signal of the doctor holding the surgical instruments is stored in a time buffer sequence. A time buffer of length L is maintained. Each time a frame of filtered data is received, the input X∈R is constructed. L×6 It then performs forward inference and outputs a category prediction.

2. The identification method as described in claim 1, characterized in that, In step one, when extracting time-domain and frequency-domain features, the time-domain to frequency-domain transformation is achieved by the real-number fast Fourier transform: X f =rfft(X) Where, X∈R L×6 Given an input sequence of length L, X f It is represented in the frequency domain.

3. The identification method as described in claim 1, characterized in that, In step two, the formula for calculating the short-time signal-to-noise ratio (SSNR) is:

4. The identification method as described in claim 3, characterized in that, In step two, the difference equation for the IIR Butterworth low-pass filter is: Where M is the filter order, b k a k These are the filter coefficients.

5. The identification method as described in claim 4, characterized in that, The formula for calculating the moving average filter is: Where W is the number of samples in the sliding window.

6. The identification method as described in claim 1, characterized in that, In step two, γ and W can be adjusted online.

7. The identification method according to any one of claims 1-6, characterized in that, Step 3 is followed by: Step 4, Evaluation and Optimization Phase: After receiving the user's termination command, the runtime T and the number of processed frames M are counted, and the actual sampling frequency f is calculated. actual Then export the original data, filtering results, and predicted labels as a CSV file by timestamp for easy subsequent analysis and visualization; otherwise, continue to step three.

8. The identification method as described in claim 7, characterized in that, Actual sampling frequency f actual The calculation formula is: f actual =M / T。

Images