Ultrasonic osteotome robot control method and device, storage medium and electronic equipment

Abstract

The invention belongs to the technical field of robot control, and particularly relates to an ultrasonic osteotome robot control method and device, a computer readable storage medium and electronic equipment. The method comprises the following steps: acquiring a multi-mode signal generated by the ultrasonic osteotome robot in the process of executing a cutting task; processing the multi-mode signal through a preset deep learning network to obtain a cutting state parameter of the ultrasonic osteotome robot; and controlling the ultrasonic osteotome robot according to the cutting state parameters. According to the method and the device, the multimode signals generated in the cutting task execution process of the ultrasonic osteotome robot can be comprehensively utilized, the cutting state parameters are obtained through deep learning network processing, the ultrasonic osteotome robot is controlled on the basis of the cutting state parameters, and the safety and the accuracy of the ultrasonic osteotome robot are effectively improved.

Description

Technical Field

[0001] This application belongs to the technical field of robot control, and particularly relates to a control method, device, computer-readable storage medium and electronic device for an ultrasonic bone scalpel robot. Background Art

[0002] With the development of technology, ultrasonic bone scalpel robots have received attention due to their high efficiency and safety in practical applications. The ultrasonic bone scalpel uses the characteristics of ultrasonic waves to cut bone tissue. Compared with traditional tools, it can achieve more precise operations and reduce damage to surrounding soft tissues. However, the existing control methods for ultrasonic bone scalpel robots still need to be improved in terms of safety and accuracy. Summary of the Invention

[0003] In view of this, embodiments of this application provide a control method, device, computer-readable storage medium and electronic device for an ultrasonic bone scalpel robot to improve the safety and accuracy of the ultrasonic bone scalpel robot.

[0004] The first aspect of the embodiments of this application provides a control method for an ultrasonic bone scalpel robot, which may include:

[0005] Obtain multi-modal signals generated during the cutting task execution of the ultrasonic bone scalpel robot;

[0006] Process the multi-modal signals through a preset deep learning network to obtain the cutting state parameters of the ultrasonic bone scalpel robot;

[0007] Control the ultrasonic bone scalpel robot according to the cutting state parameters.

[0008] In a specific implementation manner of the first aspect, before processing the multi-modal signals through a preset deep learning network, it may further include:

[0009] Preprocess the multi-modal signals to obtain the preprocessed multi-modal signals;

[0010] Correspondingly, the process of processing the multi-modal signals through a preset deep learning network to obtain the cutting state parameters of the ultrasonic bone scalpel robot may include:

[0011] Process the preprocessed multi-modal signals through the deep learning network to obtain the cutting state parameters.

[0012] In a specific implementation manner of the first aspect, the process of preprocessing the multi-modal signals to obtain the preprocessed multi-modal signals may include:

[0013] Filter the multi-modal signals to obtain the filtered multi-modal signals;

[0014] Interpolate the filtered multi-mode signal to obtain the interpolated multi-mode signal;

[0015] Perform scale normalization on the interpolated multi-mode signal to obtain the scale-normalized multi-mode signal;

[0016] Perform spectral image conversion on the scale-normalized multi-mode signal to obtain the preprocessed multi-mode signal.

[0017] In a specific implementation manner of the first aspect, the deep learning network may include a ResNet layer, an LSTM layer, and a fully connected layer;

[0018] Processing the multi-mode signal through a preset deep learning network to obtain the cutting state parameters of the ultrasonic bone scalpel robot may include:

[0019] Process the multi-mode signal through the ResNet layer to obtain first feature data;

[0020] Process the first feature data through the LSTM layer to obtain second feature data;

[0021] Process the second feature data through the fully connected layer to obtain the cutting state parameters of the ultrasonic bone scalpel robot.

[0022] In a specific implementation manner of the first aspect, the cutting state parameters may include cutting bone layer information and predicted cut-off depth; the fully connected layer may include a first fully connected layer and a second fully connected layer;

[0023] Processing the second feature data through the fully connected layer to obtain the cutting state parameters of the ultrasonic bone scalpel robot may include:

[0024] Process the second feature data through the first fully connected layer to obtain the cutting bone layer information;

[0025] Process the second feature data through the second fully connected layer to obtain the predicted cut-off depth.

[0026] In a specific implementation manner of the first aspect, the cutting state parameters may include cutting bone layer information and predicted cut-off depth; controlling the ultrasonic bone scalpel robot according to the cutting state parameters may include:

[0027] Determine the depths of the upper and lower surfaces of the bone layer according to the cutting bone layer information;

[0028] Determine the current maximum cutting depth according to the predicted cut-off depth and the depths of the upper and lower surfaces of the bone layer;

[0029] When the cutting depth is less than the current maximum cutting depth, control the ultrasonic osteotome robot to continue cutting;

[0030] When the cutting depth is greater than or equal to the current maximum cutting depth, control the ultrasonic osteotome robot to stop cutting.

[0031] In a specific implementation manner of the first aspect, determining the current maximum cutting depth according to the predicted cut-off depth and the depths of the upper and lower surfaces of the bone layer may include:

[0032] When the predicted cut-off depth is less than or equal to the depths of the upper and lower surfaces of the bone layer, determine the maximum value between the predicted cut-off depth and the existing maximum cutting depth as the current maximum cutting depth;

[0033] When the predicted cut-off depth is greater than the depths of the upper and lower surfaces of the bone layer, determine the maximum value among the existing maximum cutting depths as the current maximum cutting depth.

[0034] A second aspect of the embodiments of the present application provides a control device for an ultrasonic osteotome robot, which may include:

[0035] A signal acquisition module, configured to acquire multi-mode signals generated by the ultrasonic osteotome robot during the execution of a cutting task;

[0036] A network processing module, configured to process the multi-mode signals through a preset deep learning network to obtain the cutting state parameters of the ultrasonic osteotome robot;

[0037] A robot control module, configured to control the ultrasonic osteotome robot according to the cutting state parameters.

[0038] In a specific implementation manner of the second aspect, the control device for the ultrasonic osteotome robot may further include:

[0039] A signal preprocessing module, configured to preprocess the multi-mode signals to obtain the preprocessed multi-mode signals;

[0040] Correspondingly, the network processing module may be specifically configured to: process the preprocessed multi-mode signals through the deep learning network to obtain the cutting state parameters.

[0041] In a specific implementation manner of the second aspect, the signal preprocessing module may include:

[0042] A filtering unit, configured to filter the multi-mode signals to obtain the filtered multi-mode signals;

[0043] An interpolation unit for interpolating the filtered multi-mode signal to obtain the interpolated multi-mode signal;

[0044] A scale normalization unit for performing scale normalization on the interpolated multi-mode signal to obtain the scale-normalized multi-mode signal;

[0045] A spectrum image conversion unit for performing spectrum image conversion on the scale-normalized multi-mode signal to obtain the preprocessed multi-mode signal.

[0046] In a specific implementation manner of the second aspect, the deep learning network may include a ResNet layer, an LSTM layer, and a fully connected layer;

[0047] The network processing module may include:

[0048] A ResNet layer processing unit for processing the multi-mode signal through the ResNet layer to obtain first feature data;

[0049] An LSTM layer processing unit for processing the first feature data through the LSTM layer to obtain second feature data;

[0050] A fully connected layer processing unit for processing the second feature data through the fully connected layer to obtain the cutting state parameters of the ultrasonic bone scalpel robot.

[0051] In a specific implementation manner of the second aspect, the cutting state parameters may include cutting bone layer information and predicted cut-off depth; the fully connected layer may include a first fully connected layer and a second fully connected layer;

[0052] The fully connected layer processing unit may specifically be used for: processing the second feature data through the first fully connected layer to obtain the cutting bone layer information; processing the second feature data through the second fully connected layer to obtain the predicted cut-off depth.

[0053] In a specific implementation manner of the second aspect, the cutting state parameters include cutting bone layer information and predicted cut-off depth; the robot control module may include:

[0054] A bone layer upper and lower surface depth determination unit for determining the bone layer upper and lower surface depths according to the cutting bone layer information;

[0055] A maximum cutting depth determination unit for determining the current maximum cutting depth according to the predicted cut-off depth and the bone layer upper and lower surface depths;

[0056] The first control unit is configured to control the ultrasonic bone scalpel robot to continue cutting when the cutting depth is less than the current maximum cutting depth;

[0057] The second control unit is configured to control the ultrasonic bone scalpel robot to stop cutting when the cutting depth is greater than or equal to the current maximum cutting depth.

[0058] In a specific implementation manner of the second aspect, the maximum cutting depth determination unit may specifically be configured to: when the predicted cut-off depth is less than or equal to the upper and lower surface depths of the bone layer, determine the maximum value between the predicted cut-off depth and the existing maximum cutting depth as the current maximum cutting depth; when the predicted cut-off depth is greater than the upper and lower surface depths of the bone layer, determine the maximum value of the existing maximum cutting depths as the current maximum cutting depth.

[0059] A third aspect of the embodiments of the present application provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps of any of the above ultrasonic bone scalpel robot control methods are implemented.

[0060] A fourth aspect of the embodiments of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the steps of any of the above ultrasonic bone scalpel robot control methods are implemented.

[0061] A fifth aspect of the embodiments of the present application provides a computer program product, and when the computer program product runs on an electronic device, the electronic device is caused to execute the steps of any of the above ultrasonic bone scalpel robot control methods.

[0062] The beneficial effects of the embodiments of the present application compared with the prior art are as follows: The embodiments of the present application acquire multi-modal signals generated by the ultrasonic bone scalpel robot during the cutting task; process the multi-modal signals through a preset deep learning network to obtain the cutting state parameters of the ultrasonic bone scalpel robot; control the ultrasonic bone scalpel robot according to the cutting state parameters. Through the embodiments of the present application, the multi-modal signals generated by the ultrasonic bone scalpel robot during the cutting task can be comprehensively utilized, and the cutting state parameters are obtained through the processing of the deep learning network, and the ultrasonic bone scalpel robot is controlled based on this, effectively improving the safety and accuracy of the ultrasonic bone scalpel robot. Description of the Drawings

[0063] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0064] Figure 1 Schematic diagram of an ultrasonic bone scalpel;

[0065] Figure 2 Schematic diagram of the combined structure of cortical bone and cancellous bone;

[0066] Figure 3 Flowchart of an embodiment of a control method for an ultrasonic bone scalpel robot in an embodiment of the present application;

[0067] Figure 4 Schematic diagram of the network structure of a deep learning network;

[0068] Figure 5 Structural diagram of an embodiment of a control device for an ultrasonic bone scalpel robot in an embodiment of the present application;

[0069] Figure 6 Schematic block diagram of an electronic device in an embodiment of the present application. Detailed implementation manners

[0070] To make the invention purposes, features, and advantages of the present application more obvious and understandable, the following will clearly and completely describe the technical solutions in the embodiments of the present application in combination with the drawings in the embodiments of the present application. Obviously, the embodiments described below are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present application.

[0071] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0072] It should also be understood that the terms used in this specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this specification of the present application and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms.

[0073] It should also be further understood that the term "and / or" as used in the specification and appended claims of the present application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0074] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "once" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrase "if determined" or "if [the described condition or event] is detected" can be interpreted as meaning "once determined" or "in response to determining" or "once [the described condition or event] is detected" or "in response to detecting [the described condition or event]" depending on the context.

[0075] In addition, in the description of the present application, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0076] With the development of technology, ultrasonic bone cutting robots have received attention due to their high efficiency and safety in practical applications. Ultrasonic bone cutters utilize the characteristics of ultrasonic waves to cut bone tissue. Compared with traditional tools, they can achieve more precise operations and reduce damage to surrounding soft tissues. However, the existing control methods for ultrasonic bone cutting robots still need to be improved in terms of safety and precision.

[0077] In view of this, the embodiments of the present application provide a control method, device, computer-readable storage medium, and electronic device for an ultrasonic bone cutting robot to improve the safety and precision of the ultrasonic bone cutting robot.

[0078] In the embodiments of the present application, multi-mode signals generated during the cutting task execution process of the ultrasonic bone cutting robot can be comprehensively utilized, and the cutting state parameters can be obtained through deep learning network processing. Based on this, the ultrasonic bone cutting robot is controlled, effectively improving the safety and precision of the ultrasonic bone cutting robot.

[0079] Figure 1 Shown is a schematic diagram of an ultrasonic bone cutter. The ultrasonic bone cutter may include, but is not limited to, a tool body and a transducer. The tool body may include a tool head that contacts the target object, and this tool head serves as the executor of cutting. The shapes of the tool heads are diverse and may include, but are not limited to, various tool heads for cutting, grinding, and drilling. The transducer can be respectively connected to the tool head and the ultrasonic drive power supply, and is used to convert electrical energy into ultrasonic vibration. Among them, the ultrasonic drive power supply connection is a power supply device that can generate ultrasonic drive signals. The transducer can also be used to add expansion modules, such as connecting to a feedback circuit that collects acoustic impedance information such as current and voltage. The robot can be connected to the ultrasonic bone cutter through a fixed fixture to control its cutting.

[0080] Bone tissue can be structurally divided into cortical bone and cancellous bone, and the multimodal signals generated when the tool tip contacts cortical bone and cancellous bone are also different. There can be multiple types of bone tissue. For example, it can be cortical bone, or for another example, it can be a combined structure of cortical bone and cancellous bone as shown in Figure 2 . It can be specifically determined according to the actual situation, and the embodiments of the present application do not make specific limitations on this.

[0081] Please refer to Figure 3 . An embodiment of a method for controlling an ultrasonic bone scalpel robot in an embodiment of the present application may include:

[0082] Step S301: Obtain the multimodal signals generated during the cutting task execution by the ultrasonic bone scalpel robot.

[0083] Among them, the multimodal signals may include but are not limited to force signals and cutting depth. The force signal can be collected by a force sensor fixed on the end effector of the robot, and the cutting depth can be collected by a displacement sensor fixed on the end effector of the robot. In the embodiments of the present application, the displacement of the end effector of the robot in the cutting direction can be used as the cutting depth. The force signal and the cutting depth are both one-dimensional data that change with time in essence, and each type of data can reflect the bone condition during the cutting process to a certain extent. In the embodiments of the present application, the multimodal signals including the force signal and the cutting depth can be fused to improve the safety and accuracy of the cutting process.

[0084] Considering that there are many deficiencies in the process of collecting multimodal signals, such as complex sensor structures, large signal noise, being easily affected by external interference, and also being affected by calibration thresholds. Therefore, after obtaining the multimodal signals, the multimodal signals can also be preprocessed to obtain the preprocessed multimodal signals.

[0085] In a specific implementation manner of the embodiment of the present application, the multimode signal can be filtered to remove noise and irrelevant frequency components in the data, so as to obtain the filtered multimode signal. Since there are differences in the acquisition frequencies of multimode signals such as force signals and cutting depths, the filtered multimode signal can be interpolated by means of the time stamps of data acquisition to unify the acquisition frequencies of the multimode signals, so as to obtain the interpolated multimode signal. The interpolated multimode signal can also be subjected to scale normalization to adjust the scale of the data so that it is within a standard range, such as between 0 and 1, so as to obtain the scale-normalized multimode signal. Finally, the short-time Fourier transform (STFT) can also be used to perform spectral image conversion on the scale-normalized multimode signal to convert it into a spectral image, so as to obtain the preprocessed multimode signal. This conversion extracts time and frequency information, can make up for the irregularity of the multimode signal, and helps subsequent effective analysis. When the multimode signal is preprocessed, the multimode signal mentioned later refers to the preprocessed multimode signal.

[0086] Step S302: Process the multimode signal through a preset deep learning network to obtain the cutting state parameters of the ultrasonic bone scalpel robot.

[0087] Among them, the cutting state parameters may include but are not limited to cutting bone layer information and predicted cut-off depth. The specific network structure of the deep learning network can be flexibly set according to the actual situation, and may include but are not limited to any deep learning network structure in the prior art, and the embodiment of the present application does not make specific limitations thereto.

[0088] Since the hardness, thickness, and cutting angle of the bone layer cut each time are different, if the traditional feature extraction + threshold method is used, the robustness of penetration recognition may be poor. In order to achieve an almost-penetrating-but-not-penetrating effect when cutting the bone layer, in a specific implementation manner of the embodiment of the present application, the deep learning network may adopt a network structure combining ResNet and LSTM, and consider both spatial features and time series data for prediction. Among them, ResNet is good at processing the spatial features of image data, while LSTM is suitable for processing time series data. Therefore, combining the two can capture the features of the data more comprehensively.

[0089] As an example, Figure 4The schematic diagram of the network structure of a possible deep learning network is shown. Among them, Conv2D is a two-dimensional convolution, AdaptiveAvgPool 2D is an adaptive pooling layer, Flatten is a flattening layer, Fully connected is a fully connected layer, and Softmax is an activation function module. First, the multimodal signal can be processed through each layer of two-dimensional convolution in the ResNet layer, and can also be processed through the adaptive pooling layer and the flattening layer, so as to obtain the first feature data. Then, the first feature data can be processed through the LSTM layer to obtain the second feature data. Finally, the second feature data can be processed through the fully connected layer to obtain the cutting state parameters of the ultrasonic bone scalpel robot. Among them, the fully connected layer can include a first fully connected layer and a second fully connected layer. Specifically, the second feature data can be processed through the first fully connected layer, and can also be processed through the activation function module to obtain the cutting bone layer information. The second feature data can be processed through the second fully connected layer to obtain the predicted cut-off depth.

[0090] In order to enable the deep learning network to accurately predict the cutting state parameters, it can be pre-trained with a sufficient number of training samples in advance. Each training sample includes a set of multimodal signal samples and corresponding cutting state parameter labels. Taking the multimodal signal samples of the training samples as input and the corresponding cutting state parameter labels as the expected output, the deep learning network is trained to obtain the trained deep learning network.

[0091] During the training process, for each training sample, the deep learning network can be used to process the multimodal signal samples of the training sample to obtain the actual output of the training sample. Then, a preset loss function can be used to calculate the training loss value according to the expected output and the actual output in the training sample. In the embodiments of the present application, any loss function in the prior art can be selected according to the actual situation to calculate the training loss value, and the embodiments of the present application do not make specific limitations in this regard.

[0092] After calculating the training loss value, the model parameters of the deep learning network can be adjusted according to the training loss value. In the embodiments of the present application, it is assumed that in the initial state, the model parameters of the deep learning network are W1. The training loss value is backpropagated to modify the model parameters W1 of the deep learning network, and the modified model parameters W2 are obtained. After modifying the parameters, continue to execute the next training process. In this training process, recalculate the training loss value, backpropagate the training loss value to modify the model parameters W2 of the deep learning network, and obtain the modified model parameters W3, and so on. Repeat the above process continuously. Each training process can modify the model parameters until the preset training conditions are met. Among them, the training conditions can be that the number of training times reaches the preset number threshold, and the number threshold can be set according to the actual situation. For example, it can be set to thousands, tens of thousands, hundreds of thousands or even larger values; the training conditions can also be that the deep learning network converges; since it is possible that the number of training times has not reached the number threshold, but the deep learning network has already converged, which may lead to unnecessary repeated work; or the deep learning network cannot converge all the time, which may lead to an infinite loop and the training process cannot end. Based on the above two situations, the training conditions can also be that the number of training times reaches the number threshold or the deep learning network converges. When the training conditions are met, the trained deep learning network can be obtained.

[0093] Through the above process, the multi-modal signal samples of the training samples and the corresponding cutting state parameter labels are used as the learning objects of the deep learning network. After the training process, the deep learning network can establish a mapping relationship between the multi-modal signal samples and the cutting state parameter labels. Therefore, when facing a new multi-modal signal, the corresponding cutting state parameters can also be obtained according to this mapping relationship.

[0094] After completing the training of the deep learning network, the deep learning network can be used to process real-time multi-modal signals, so as to obtain the corresponding cutting state parameters.

[0095] Step S303: Control the ultrasonic bone scalpel robot according to the cutting state parameters.

[0096] In a specific implementation manner of the embodiments of the present application, the depths of the upper and lower surfaces of the bone layer can be determined according to the cutting bone layer information, and the current maximum cutting depth can be determined according to the predicted cut-off depth and the depths of the upper and lower surfaces of the bone layer. Among them, the depth of the upper and lower surfaces of the bone layer is the difference between the coordinate values of the upper and lower surfaces of the bone layer. When the predicted cut-off depth is less than or equal to the depth of the upper and lower surfaces of the bone layer, the maximum value between the predicted cut-off depth and the existing maximum cutting depth can be determined as the current maximum cutting depth, as shown in the following formula: D max = max(D max , D pre ), where D maxis the maximum cutting depth, D pre is the predicted cut-off depth; when the predicted cut-off depth is greater than the depths of the upper and lower surfaces of the bone layer, the maximum value among the existing maximum cutting depths can be determined as the current maximum cutting depth, i.e., the original D is maintained max unchanged.

[0097] Then, the cutting depth can be compared with the current maximum cutting depth. When the cutting depth is less than the current maximum cutting depth, the ultrasonic bone scalpel robot can be controlled to continue cutting and the Figure 3 shown control process can be re-executed, i.e., a new multi-modal signal is acquired again, the new cutting state parameters are obtained through processing by the deep learning network, and the ultrasonic bone scalpel robot is controlled according to the new cutting state parameters; when the cutting depth is greater than or equal to the current maximum cutting depth, the ultrasonic bone scalpel robot can be controlled to stop cutting.

[0098] In summary, the embodiments of the present application acquire the multi-modal signals generated by the ultrasonic bone scalpel robot during the cutting task; process the multi-modal signals through a preset deep learning network to obtain the cutting state parameters of the ultrasonic bone scalpel robot; and control the ultrasonic bone scalpel robot according to the cutting state parameters. Through the embodiments of the present application, the multi-modal signals generated by the ultrasonic bone scalpel robot during the cutting task can be comprehensively utilized, and the cutting state parameters are obtained through processing by the deep learning network, and the ultrasonic bone scalpel robot is controlled based on this, effectively improving the safety and accuracy of the ultrasonic bone scalpel robot.

[0099] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0100] Corresponding to the ultrasonic bone scalpel robot control method described in the above embodiments, Figure 5 shows a structural diagram of an embodiment of a control device for an ultrasonic bone scalpel robot provided by an embodiment of the present application.

[0101] In this embodiment, a control device for an ultrasonic bone scalpel robot may include:

[0102] A signal acquisition module 501, configured to acquire multi-modal signals generated by the ultrasonic bone scalpel robot during the cutting task;

[0103] A network processing module 502, configured to process the multi-modal signals through a preset deep learning network to obtain the cutting state parameters of the ultrasonic bone scalpel robot;

[0104] A robot control module 503, configured to control the ultrasonic bone scalpel robot according to the cutting state parameters.

[0105] In a specific implementation manner of the embodiment of the present application, the ultrasonic osteotome robot control device may further include:

[0106] A signal preprocessing module, configured to preprocess the multimode signal to obtain the preprocessed multimode signal;

[0107] Correspondingly, the network processing module may specifically be configured to: process the preprocessed multimode signal through the deep learning network to obtain the cutting state parameters.

[0108] In a specific implementation manner of the embodiment of the present application, the signal preprocessing module may include:

[0109] A filtering unit, configured to filter the multimode signal to obtain the filtered multimode signal;

[0110] An interpolation unit, configured to interpolate the filtered multimode signal to obtain the interpolated multimode signal;

[0111] A scale normalization unit, configured to perform scale normalization on the interpolated multimode signal to obtain the scale-normalized multimode signal;

[0112] A spectrum image conversion unit, configured to perform spectrum image conversion on the scale-normalized multimode signal to obtain the preprocessed multimode signal.

[0113] In a specific implementation manner of the embodiment of the present application, the deep learning network may include a ResNet layer, an LSTM layer, and a fully connected layer;

[0114] The network processing module may include:

[0115] A ResNet layer processing unit, configured to process the multimode signal through the ResNet layer to obtain first feature data;

[0116] An LSTM layer processing unit, configured to process the first feature data through the LSTM layer to obtain second feature data;

[0117] A fully connected layer processing unit, configured to process the second feature data through the fully connected layer to obtain the cutting state parameters of the ultrasonic osteotome robot.

[0118] In a specific implementation manner of the embodiment of the present application, the cutting state parameters may include cutting bone layer information and predicted cut-off depth; the fully connected layer may include a first fully connected layer and a second fully connected layer;

[0119] The fully connected layer processing unit may be specifically configured to: process the second feature data through the first fully connected layer to obtain the cutting bone layer information; process the second feature data through the second fully connected layer to obtain the predicted cut-off depth.

[0120] In a specific implementation manner of the embodiment of the present application, the cutting state parameter includes the cutting bone layer information and the predicted cut-off depth; the robot control module may include:

[0121] A bone layer upper and lower surface depth determination unit, configured to determine the bone layer upper and lower surface depths according to the cutting bone layer information;

[0122] A maximum cutting depth determination unit, configured to determine the current maximum cutting depth according to the predicted cut-off depth and the bone layer upper and lower surface depths;

[0123] A first control unit, configured to control the ultrasonic bone knife robot to continue cutting when the cutting depth is less than the current maximum cutting depth;

[0124] A second control unit, configured to control the ultrasonic bone knife robot to stop cutting when the cutting depth is greater than or equal to the current maximum cutting depth.

[0125] In a specific implementation manner of the embodiment of the present application, the maximum cutting depth determination unit may be specifically configured to: when the predicted cut-off depth is less than or equal to the bone layer upper and lower surface depths, determine the maximum value between the predicted cut-off depth and the existing maximum cutting depth as the current maximum cutting depth; when the predicted cut-off depth is greater than the bone layer upper and lower surface depths, determine the maximum value among the existing maximum cutting depths as the current maximum cutting depth.

[0126] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described devices, modules, and units can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0127] In the above embodiments, the descriptions of the respective embodiments have their own emphases. For parts not detailed or recorded in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0128] Figure 6 The schematic block diagram of an electronic device provided by an embodiment of the present application is shown. For the sake of convenience of description, only parts related to the embodiment of the present application are shown.

[0129] Such as Figure 6As shown, the electronic device 6 of this embodiment includes: a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and executable on the processor 60. When the processor 60 executes the computer program 62, the steps in the above-mentioned embodiments of each ultrasonic bone scalpel robot control method are implemented, such as Figure 3 The steps S301 to S303 shown. Alternatively, when the processor 60 executes the computer program 62, the functions of each module / unit in the above-mentioned device embodiments are implemented, such as Figure 5 The functions of the modules 501 to 503 shown.

[0130] Exemplarily, the computer program 62 can be divided into one or more modules / units. The one or more modules / units are stored in the memory 61 and executed by the processor 60 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of completing specific functions, and these instruction segments are used to describe the execution process of the computer program 62 in the electronic device 6.

[0131] The electronic device 6 may include, but is not limited to, computing devices such as desktop computers, notebooks, palm computers, servers, and robots. Those skilled in the art can understand that Figure 6 These are merely examples of the electronic device 6 and do not constitute a limitation on the electronic device 6. It may include more or fewer components than shown in the figure, or combine certain components, or have different components. For example, the electronic device 6 may further include input / output devices, network access devices, buses, etc.

[0132] The processor 60 may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0133] The memory 61 may be an internal storage unit of the electronic device 6, such as a hard disk or memory of the electronic device 6. The memory 61 may also be an external storage device of the electronic device 6, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the electronic device 6. Further, the memory 61 may also include both the internal storage unit and the external storage device of the electronic device 6. The memory 61 is used to store the computer program and other programs and data required by the electronic device 6. The memory 61 may also be used to temporarily store the data that has been output or will be output.

[0134] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of the present application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiment and will not be elaborated herein.

[0135] In the above embodiments, the descriptions of the respective embodiments have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0136] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or by a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0137] In the embodiments provided in the present application, it should be understood that the disclosed device / electronic device and method can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling, direct coupling, or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in electrical, mechanical, or other forms.

[0138] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0139] In addition, in each embodiment of the present application, the functional units can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0140] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above method embodiments of the present application, it can also be completed by a computer program instructing relevant hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file, or some intermediate form, etc. The computer-readable storage medium can include: any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.

[0141] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included within the protection scope of the present application.

Claims

1. A method for controlling an ultrasonic bone cutting robot, characterized in that: include: Acquire the multi-mode signal generated by the ultrasonic bone cutting robot during the cutting task; The multi-mode signal is processed by a preset deep learning network to obtain the cutting state parameters of the ultrasonic bone knife robot; The ultrasonic bone cutting robot is controlled according to the cutting state parameters.

2. The ultrasonic bone cutting robot control method according to claim 1, characterized in that: Before the multi-mode signal is processed by a preset deep learning network, the method further includes: Preprocessing the multimode signal to obtain the preprocessed multimode signal; Accordingly, the multi-mode signal is processed by a preset deep learning network to obtain the cutting state parameters of the ultrasonic bone knife robot, including: The preprocessed multimode signal is processed by the deep learning network to obtain the cutting state parameter.

3. The ultrasonic bone cutting robot control method according to claim 2, characterized in that: The preprocessing of the multimode signal to obtain the preprocessed multimode signal includes: Filtering the multimode signal to obtain the filtered multimode signal; Interpolating the filtered multimode signal to obtain the interpolated multimode signal; Normalizing the interpolated multimode signal to obtain the normalized multimode signal; The scale-normalized multimode signal is subjected to spectrum image conversion to obtain the preprocessed multimode signal.

4. The ultrasonic bone cutting robot control method according to claim 1, characterized in that: The deep learning network includes a ResNet layer, an LSTM layer and a fully connected layer; The multi-mode signal is processed by a preset deep learning network to obtain the cutting state parameters of the ultrasonic bone knife robot, including: Processing the multimode signal through the ResNet layer to obtain first feature data; Processing the first feature data through the LSTM layer to obtain second feature data; The second feature data is processed by the fully connected layer to obtain the cutting state parameters of the ultrasonic bone knife robot.

5. The ultrasonic bone cutting robot control method according to claim 4, characterized in that: The cutting state parameters include cutting bone layer information and predicted cutoff depth; the fully connected layer includes a first fully connected layer and a second fully connected layer; The step of processing the second feature data through the fully connected layer to obtain the cutting state parameters of the ultrasonic bone cutting robot includes: Processing the second feature data through the first fully connected layer to obtain the cut bone layer information; The second feature data is processed by the second fully connected layer to obtain the predicted cutoff depth.

6. The ultrasonic osteotome robot control method according to any one of claims 1 to 5, characterized in that: The cutting state parameters include cutting bone layer information and predicted cut-off depth; and controlling the ultrasonic bone knife robot according to the cutting state parameters includes: Determine the depth of the upper and lower surfaces of the bone layer according to the cut bone layer information; Determining a current maximum cutting depth according to the predicted cutoff depth and the depths of the upper and lower surfaces of the bone layer; When the cutting depth is less than the current maximum cutting depth, controlling the ultrasonic bone knife robot to continue cutting; When the cutting depth is greater than or equal to the current maximum cutting depth, the ultrasonic bone knife robot is controlled to stop cutting.

7. The ultrasonic bone cutting robot control method according to claim 6, characterized in that: Determining the current maximum cutting depth according to the predicted cutoff depth and the depths of the upper and lower surfaces of the bone layer includes: In the case where the predicted cutoff depth is less than or equal to the depths of the upper and lower surfaces of the bone layer, determining the maximum value of the predicted cutoff depth and the existing maximum cutting depth as the current maximum cutting depth; In the case where the predicted cutoff depth is greater than the depths of the upper and lower surfaces of the bone layer, the maximum value of the existing maximum cutting depths is determined as the current maximum cutting depth.

8. An ultrasonic bone cutting robot control device, characterized in that: include: A signal acquisition module is used to acquire the multi-mode signal generated by the ultrasonic bone cutting robot during the cutting task; A network processing module, used to process the multi-mode signal through a preset deep learning network to obtain the cutting state parameters of the ultrasonic bone knife robot; A robot control module is used to control the ultrasonic bone knife robot according to the cutting state parameters.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the ultrasonic bone cutting robot control method according to any one of claims 1 to 7 are implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the ultrasonic bone cutting robot control method according to any one of claims 1 to 7 are implemented.

Images