Robotic arm control method based on deep learning, and surgical robot and storage medium

Abstract

Provided are a robotic arm control method based on deep learning, and a surgical robot and a storage medium. The method comprises: acquiring an ultrasound image to be detected, wherein said ultrasound image represents an image in a state in which a robotic arm reaches a body surface of a target object (S110); inputting said ultrasound image into a pre-trained deep learning model for image processing to obtain an image recognition result (S120); determining initial movement trajectory information of the robotic arm (S130); on the basis of the image recognition result, the initial movement trajectory information and a preset rule, determining a target movement instruction (S140); and on the basis of the target movement instruction, controlling the movement of the robotic arm (S150). The robotic arm control method based on deep learning controls the movement of the robotic arm by means of the ultrasound image and a deep learning result based on the ultrasound image. Compared with a method for controlling the movement of a robotic arm on the basis of a preset force and preset path planning, the robotic arm control method can modify a movement path of the robotic arm on the basis of actual conditions, thereby effectively improving the control accuracy of the robotic arm of the surgical robot.

Description

Deep learning-based robotic arm control method, surgical robot, and storage medium Technical Field

[0001] The present application relates to, but is not limited to, the field of medical equipment control technology, and in particular to a deep learning-based robotic arm control method, surgical robot, and storage medium. Background Art

[0002] Existing surgical robots control the movement of the robotic arm by setting forces and given coordinates based on a pre-set path plan, aiming for the robot to land at a target location on the human body. However, the human body has a limited range of tolerance, and when different people experience the same force, it is difficult to determine the appropriate force to stop the robotic arm. Furthermore, if the target location is located according to the pre-set path plan, the path needs to be replanned even with the slightest movement of the human body, resulting in low accuracy and efficiency in the robotic arm's control.

[0003] Summary of the Invention

[0004] The embodiments of the present application provide a deep learning-based robotic arm control method, a surgical robot, and a storage medium, which can effectively improve the control accuracy of the surgical robot's robotic arm.

[0005] In a first aspect, an embodiment of the present application provides a robotic arm control method based on deep learning, comprising:

[0006] Acquire an ultrasonic image to be detected, wherein the ultrasonic image to be detected represents an image of a state in which the robotic arm reaches the body surface of the target object;

[0007] Inputting the ultrasound image to be detected into a pre-trained deep learning model for image processing to obtain an image recognition result;

[0008] Determining initial motion trajectory information of the robotic arm;

[0009] Determining a target motion instruction based on the image recognition result, the initial motion trajectory information, and a preset rule;

[0010] The movement of the robotic arm is controlled according to the target movement instruction.

[0011] In some embodiments, determining the target motion instruction based on the image recognition result, the initial motion trajectory information, and a preset rule includes:

[0012] When the image recognition result does not meet the preset conditions, determining the reference force and reference landing point information corresponding to the ultrasonic image to be detected;

[0013] The target motion instruction is determined according to the reference force, the reference landing point information and the initial motion trajectory information.

[0014] In some embodiments, the initial motion trajectory information includes a target landing point coordinate range, and determining the target motion instruction based on the reference force, the reference landing point information, and the initial motion trajectory information includes:

[0015] When the reference landing point coordinates exceed the target landing point coordinate range, the target motion instruction is determined according to a reference distance value and reference direction information, wherein the reference distance value is the distance between the reference landing point coordinates and a boundary point of the target landing point coordinate range, and the reference direction information is the direction vector of the reference landing point coordinates from the nearest boundary point of the target landing point coordinate range;

[0016] or,

[0017] When the reference force is less than the target force, the target motion instruction is determined according to the reference force and the target force, where the reference force is the force applied by the robotic arm to the body surface of the target object.

[0018] In some embodiments, the deep learning model includes a first convolutional layer, a maximum pooling layer, a second convolutional layer, and a fully connected layer connected in sequence, and inputting the ultrasound image to be detected into a pre-trained deep learning model for image processing to obtain an image recognition result includes:

[0019] Inputting the ultrasonic image to be detected into the first convolutional layer for performing a convolution operation to obtain a first intermediate image;

[0020] Inputting the first intermediate image into the maximum pooling layer for sampling processing to obtain a second intermediate image;

[0021] Inputting the second intermediate image into the second convolutional layer for performing a convolution operation to obtain a third intermediate image;

[0022] The third intermediate image is input into the second convolutional layer for image processing to obtain the image recognition result.

[0023] In some embodiments, the deep learning model further includes a first BN layer, a second BN layer, a first activation layer, and a second activation layer, wherein the first BN layer and the first activation layer are respectively connected to the first convolutional layer, and the second BN layer and the second activation layer are respectively connected to the first convolutional layer.

[0024] In some embodiments, after controlling the movement of the robotic arm according to the target movement instruction, the method further includes:

[0025] Reacquiring a new ultrasound image to be detected, inputting the new ultrasound image to be detected into the deep learning model for image processing, and obtaining a new image recognition result;

[0026] When the new image recognition result meets the preset condition, the robotic arm is controlled to stop moving.

[0027] In some embodiments, before inputting the ultrasound image to be detected into a pre-trained deep learning model for image processing to obtain an image recognition result, the method further includes:

[0028] Get the preset image preprocessing rules;

[0029] The ultrasonic image to be detected is preprocessed according to the image preprocessing rule to obtain a preprocessed ultrasonic image to be detected.

[0030] In a second aspect, an embodiment of the present application provides a surgical robot, comprising:

[0031] An image acquisition module, the image acquisition module is used to acquire an ultrasonic image to be detected, wherein the ultrasonic image to be detected represents an image when the robotic arm reaches the body surface of the target object;

[0032] An image processing module is used to input the ultrasound image to be detected into a pre-trained deep learning model for image processing to obtain an image recognition result;

[0033] An initial motion trajectory information acquisition module, the initial motion trajectory information acquisition module is used to determine the initial motion trajectory information of the robotic arm;

[0034] a target motion instruction acquisition module, configured to determine a target motion instruction based on the image recognition result, the initial motion trajectory information, and a preset rule;

[0035] A motion control module is used to control the movement of the robotic arm according to the target motion instruction.

[0036] In a third aspect, an embodiment of the present application provides a surgical robot comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the deep learning-based robotic arm control method as described in the first aspect is implemented.

[0037] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the deep learning-based robotic arm control method as described in the first aspect.

[0038] The embodiment of the present application provides a deep learning-based robotic arm control method, surgical robot, and storage medium, the method comprising: obtaining an ultrasonic image to be detected, the ultrasonic image to be detected representing an image of the robotic arm reaching the surface of the target object; inputting the ultrasonic image to be detected into a pre-trained deep learning model for image processing to obtain an image recognition result; determining the initial motion trajectory information of the robotic arm; determining a target motion instruction based on the image recognition result, the initial motion trajectory information, and a preset rule; and controlling the movement of the robotic arm according to the target motion instruction. The embodiment of the present application controls the movement of the robotic arm using ultrasonic images and deep learning results based on ultrasonic images. Compared with the prior art, which only controls the movement of the robotic arm based on preset forces and preset path planning, the motion path of the robotic arm can be changed according to actual conditions, thereby effectively improving the control accuracy of the surgical robot robotic arm. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings are used to provide a further understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0040] FIG1 is a flowchart of a deep learning-based robotic arm control method according to an embodiment of the present application;

[0041] FIG2 is a flowchart of steps for determining a target motion instruction according to another embodiment of the present application;

[0042] FIG3 is a flowchart of steps for determining a target motion instruction according to another embodiment of the present application;

[0043] FIG4 is a flowchart of steps for obtaining image recognition results provided by another embodiment of the present application;

[0044] FIG5 is a flowchart of a method for controlling a robotic arm according to another embodiment of the present invention;

[0045] FIG6 is a flowchart of steps for image preprocessing of an ultrasound image to be detected provided by another embodiment of the present application;

[0046] FIG7 is a schematic structural diagram of a robotic arm provided by another embodiment of the present application;

[0047] FIG8 is a schematic diagram of modules of a surgical robot provided by another embodiment of the present application;

[0048] FIG9 is a hardware schematic diagram of a surgical robot provided in another embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0050] It is understood that although the device schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device or the sequence in the flowcharts. The terms "first," "second," and the like in the specification, claims, or accompanying drawings are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0051] In the description of this application, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they cannot be understood as limitations on this application.

[0052] In the description of this application, unless otherwise clearly defined, terms such as setting, installing, and connecting should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in this application based on the specific content of the technical solution.

[0053] Existing surgical robots control the movement of the robotic arm by setting forces and given coordinates based on a pre-set path plan, aiming for the robot to land at a target location on the human body. However, the human body has a limited range of tolerance, and when different people experience the same force, it is difficult to determine the appropriate force to stop the robotic arm. Furthermore, if the target location is located according to the pre-set path plan, the path needs to be replanned even with the slightest movement of the human body, resulting in low accuracy and efficiency in the robotic arm's control.

[0054] To solve the above-mentioned problems, the embodiments of the present application provide a deep learning-based robotic arm control method, a surgical robot, and a storage medium, the method comprising: obtaining an ultrasonic image to be detected, the ultrasonic image to be detected representing an image of the robotic arm reaching the surface of the target object; inputting the ultrasonic image to be detected into a pre-trained deep learning model for image processing to obtain an image recognition result; determining the initial motion trajectory information of the robotic arm; determining a target motion instruction based on the image recognition result, the initial motion trajectory information, and a preset rule; and controlling the movement of the robotic arm according to the target motion instruction. The embodiments of the present application control the movement of the robotic arm with ultrasonic images and deep learning results based on ultrasonic images. Compared with the prior art scheme that controls the movement of the robotic arm only based on preset forces and preset path planning, the motion path of the robotic arm can be changed according to actual conditions, thereby effectively improving the control accuracy of the surgical robot robotic arm.

[0055] The embodiments of the present application are further described below with reference to the accompanying drawings.

[0056] As shown in FIG1 , FIG1 is a flowchart of a deep learning-based robotic arm control method provided by an embodiment of the present application. The embodiment of the present application provides a deep learning-based robotic arm control method, which includes but is not limited to the following steps:

[0057] Step S110, obtaining an ultrasonic image to be detected, where the ultrasonic image to be detected represents an image of the robotic arm reaching the body surface of the target object;

[0058] Step S120: inputting the ultrasound image to be detected into a pre-trained deep learning model for image processing to obtain an image recognition result;

[0059] Step S130, determining the initial motion trajectory information of the robotic arm;

[0060] Step S140, determining a target motion instruction based on the image recognition result, the initial motion trajectory information, and a preset rule;

[0061] Step S150: Control the movement of the robotic arm according to the target movement instruction.

[0062] It should be noted that the embodiments of the present application do not limit the specific structure for obtaining the ultrasonic image to be detected. Referring to Figure 7, the display screen 710 can be installed on a preset position of the robotic arm 700 of the surgical robot through the mounting base 730 and the positioning workpiece 740. An ultrasonic probe 750 is installed at one end of the robotic arm 700, and the display screen 710 is communicatively connected with the ultrasonic probe 750. The display screen 710 can display the ultrasonic image to be detected obtained by the ultrasonic probe 750 in real time. At the same time, the robotic arm 700 is also provided with an adjustable guide structure 720, which can move based on the target motion instruction issued by the surgical robot.

[0063] It is understandable that since the existing surgical robots only use force and given coordinates of path planning to make the robotic arm stop on the surface of the target object, slight movement of the target object may cause deviation in the insertion of the needle according to the initial path planning, resulting in inaccurate landing point of the robotic arm during the interventional surgery, affecting the surgical effect. Based on this, the embodiment of the present application obtains an ultrasonic image to be detected, which represents the image of the robotic arm when it reaches the surface of the target object. The ultrasonic image to be detected in the embodiment of the present application can be an ultrasound image of the liver carrying the positioning information of the needle port of the robotic arm. There is no restriction on the specific type of the ultrasonic image to be detected. The ultrasonic image to be detected is input into a pre-trained deep learning model for image processing to obtain an image recognition result, determine the initial motion trajectory information of the robotic arm, determine the target motion instruction based on the image recognition result, the initial motion trajectory information and the preset rules, and control the movement of the robotic arm according to the target motion instruction. The quality and effect of the ultrasonic image to be detected are judged by a trained deep learning model, and the image recognition results based on the deep learning model are used to assist the movement of the robotic arm (pressing down, rotating or stopping). Compared with the existing technology, which only controls the movement of the robotic arm according to the preset force and preset path planning, it can obtain the internal structure image of the target object and locate the needle port of the robotic arm in real time, and change the movement path of the robotic arm according to the actual situation, thereby effectively improving the control accuracy of the surgical robot robotic arm. The application scenario of the embodiment of the present application is puncture interventional surgery. During the puncture interventional surgery, the method provided by the embodiment of the present application can show the surgeon the internal situation of the puncture needle entering the target area in real time, assist the surgeon in dealing with various complex surgical situations encountered in the clinic, improve the positioning and guidance flexibility and guidance accuracy of the interventional surgery, reduce the operational difficulty of the interventional surgery, and improve the quality and safety of the surgery.

[0064] It should be noted that the embodiment of the present application does not limit the specific size of the ultrasound image to be detected, and it can be an ultrasound image with an image size of 512*512 and a channel number of 3.

[0065] As shown in FIG2 , in some embodiments, step S140 in the embodiment of FIG1 includes but is not limited to the following steps:

[0066] Step S210: When the image recognition result does not meet the preset conditions, determining the reference force and reference landing point information corresponding to the ultrasonic image to be detected;

[0067] Step S220 , determining a target motion instruction according to the reference force, the reference landing point information and the initial motion trajectory information.

[0068] It can be understood that the image recognition result includes a first classification result and a second classification result. The first classification result and the second classification result are divided based on preset conditions. The first classification result indicates that the ultrasound image to be detected is classified as a poor landing point image, that is, the ultrasound image to be detected cannot clearly display the liver structure and tumor location; the second classification result indicates that the ultrasound image to be detected is classified as a good landing point image, that is, the ultrasound image to be detected can clearly display the liver structure and tumor location. When the image recognition result does not meet the preset conditions, that is, the image recognition result is determined to be the first classification result, the reference force and reference landing point information corresponding to the ultrasound image to be detected are determined, and the target motion instruction is determined according to the reference force, reference landing point information and initial motion trajectory information, so that the initial motion trajectory information can be intelligently adjusted to change the motion path of the robotic arm.

[0069] In some embodiments, the initial motion trajectory information includes a target landing point coordinate range. Referring to FIG3 , step S220 in the embodiment of FIG2 includes but is not limited to the following steps:

[0070] Step S310: When the reference landing point coordinates exceed the target landing point coordinate range, a target motion instruction is determined based on a reference distance value and reference direction information, where the reference distance value is the distance between the reference landing point coordinates and a boundary point of the target landing point coordinate range, and the reference direction information is the direction vector of the closest boundary point from the reference landing point coordinates to the target landing point coordinate range.

[0071] Step S320 : When the reference force is less than the target force, a target motion instruction is determined according to the reference force and the target force, where the reference force is the force applied by the robotic arm to the surface of the target object.

[0072] It can be understood that when the image recognition result does not meet the preset conditions, the landing point coordinates and reference force of the current robotic arm are determined, that is, the reference landing point coordinates and reference force are obtained, and based on the target landing point coordinate range and the target force as a benchmark, the target motion instruction is generated in combination with the reference landing point coordinates and the reference force, and the movement of the robotic arm is controlled so that the robotic arm moves to a position corresponding to the preset conditions. The specific method is: when the reference landing point coordinates exceed the target landing point coordinate range, the target motion instruction is determined according to the reference distance value and the reference direction information, wherein the reference distance value is the distance value between the reference landing point coordinate and the boundary point of the target landing point coordinate range, and the reference direction information is the direction vector of the reference landing point coordinate from the nearest boundary point of the target landing point coordinate range; when the reference force is less than the target force, the target motion instruction is determined according to the reference force and the target force, and the reference force is the force applied by the robotic arm to the surface of the target object.

[0073] In some embodiments, the deep learning model includes a first convolutional layer, a maximum pooling layer, a second convolutional layer, and a fully connected layer connected in sequence. Referring to FIG4 , step S120 in the embodiment of FIG1 includes but is not limited to the following steps:

[0074] Step S410: Input the ultrasonic image to be detected into the first convolutional layer for convolution operation to obtain a first intermediate image;

[0075] Step S420: Input the first intermediate image to the maximum pooling layer for sampling processing to obtain a second intermediate image;

[0076] Step S430: Input the second intermediate image into the second convolutional layer for convolution operation to obtain a third intermediate image;

[0077] Step S440: Input the third intermediate image into the second convolutional layer for image processing to obtain an image recognition result.

[0078] It should be noted that the embodiments of the present application are not limited to specific deep learning models, and may be a logistic regression model, a support vector machine, a random forest model, etc., and those skilled in the art may select one according to actual circumstances.

[0079] It should be noted that the embodiments of the present application do not limit the specific sizes of the first convolutional layer, the second convolutional layer, the maximum pooling layer, and the fully connected layer. The first convolutional layer and the second convolutional layer can both be convolutional layers with a convolution kernel size of 3*3 and a stride of 2. The maximum pooling layer can be a pooling layer with a convolution kernel size of 3*3 and a stride of 2. The number of channels of the fully connected layer can be 2.

[0080] It should be noted that the deep learning model also includes a first BN layer, a second BN layer, a first activation layer and a second activation layer. The first BN layer and the first activation layer are respectively connected to the first convolutional layer, and the second BN layer and the second activation layer are respectively connected to the first convolutional layer. It can be understood that the BN layer connected after the convolutional layer can prevent gradient explosion and gradient disappearance during the application of the model, and the activation layer connected after the convolutional layer can enhance the expressive ability of the network model and improve the accuracy of the deep learning model.

[0081] In some embodiments, referring to FIG5 , after executing step S150 in the embodiment of FIG1 , the deep learning-based robotic arm control method provided in the embodiment of the present application further includes but is not limited to the following steps:

[0082] Step S510: reacquire a new ultrasound image to be detected, input the new ultrasound image to be detected into the deep learning model for image processing, and obtain a new image recognition result;

[0083] Step S520: When the new image recognition result meets the preset conditions, the robotic arm is controlled to stop moving.

[0084] It can be understood that when the image recognition result does not meet the preset conditions, and in the process of determining the target motion instruction to control the movement of the robotic arm based on the image recognition result, the initial motion trajectory information and the preset rules, a new ultrasound image to be detected is reacquired, and the new ultrasound image to be detected is input into the deep learning model for image processing to obtain a new image recognition result. When the new image recognition result meets the preset conditions, refer to the description of the above embodiment, that is, the new ultrasound image to be detected can clearly show the liver structure and tumor location, the robotic arm reaches the target position, and the robotic arm is controlled to stop moving.

[0085] In some embodiments, referring to FIG6 , before executing step S120 , the deep learning-based robotic arm control method of the embodiment of the present application further includes but is not limited to the following steps:

[0086] Step S610, obtaining preset image preprocessing rules;

[0087] Step S620 , performing image preprocessing on the ultrasonic image to be detected according to an image preprocessing rule to obtain a preprocessed ultrasonic image to be detected.

[0088] It is understandable that the main purpose of image preprocessing for the ultrasonic image to be detected is to eliminate irrelevant information in the ultrasonic image to be detected, restore useful real information, enhance the detectability of relevant information and simplify the data to the maximum extent, thereby improving the reliability of subsequent applications of the ultrasonic image to be detected.

[0089] It should be noted that the embodiments of the present application do not limit the specific method for preprocessing the ultrasonic image to be detected. It may be image random number rotation processing, normalization processing or image binarization processing, etc. The technical personnel in this field can select the specific method of image preprocessing according to the actual situation.

[0090] In addition, referring to FIG8 , an embodiment of the present application further provides a surgical robot 800 , which includes:

[0091] An image acquisition module 810 is used to acquire an ultrasonic image to be detected, where the ultrasonic image to be detected represents an image of the robot arm reaching the surface of the target object;

[0092] The image processing module 820 is used to input the ultrasound image to be detected into a pre-trained deep learning model for image processing to obtain an image recognition result;

[0093] An initial motion trajectory information acquisition module 830 is used to determine the initial motion trajectory information of the robotic arm;

[0094] The target motion instruction acquisition module 840 is used to determine the target motion instruction according to the image recognition result, the initial motion trajectory information and the preset rules;

[0095] The motion control module 850 is used to control the motion of the robotic arm according to the target motion instruction.

[0096] It should be noted that the specific implementation of the surgical robot 800 is basically the same as the specific embodiment of the above-mentioned deep learning-based robotic arm control method, and will not be repeated here.

[0097] In addition, refer to Figure 9, which is a hardware schematic diagram of a surgical robot provided by another embodiment of the present application. One embodiment of the present application also provides a surgical robot 900, which includes: a memory 910, a processor 920, and a computer program stored in the memory 910 and executable on the processor 920.

[0098] The processor 920 and the memory 910 may be connected via a bus or other means.

[0099] The non-transient software programs and instructions required to implement the deep learning-based robotic arm control method of the above-mentioned embodiment are stored in the memory 910. When executed by the processor 920, the deep learning-based robotic arm control method in the above-mentioned embodiment is executed, for example, the method steps S110 to S150 in Figure 1, the method steps S210 to S220 in Figure 2, the method steps S310 to S320 in Figure 3, the method steps S410 to S440 in Figure 4, the method steps S510 to S520 in Figure 5 and the method steps S610 to S620 in Figure 6 described above are executed.

[0100] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0101] In addition, one embodiment of the present application further provides a computer-readable storage medium storing computer-executable instructions. The computer-executable instructions are executed by a processor or controller, for example, by a processor in the embodiment of the puncture surgical robot 900, so that the processor can execute the deep learning-based robotic arm control method in the above embodiment, for example, executing the method steps S110 to S150 in FIG1, the method steps S210 to S220 in FIG2, the method steps S310 to S320 in FIG3, the method steps S410 to S440 in FIG4, the method steps S510 to S520 in FIG5, and the method steps S610 to S620 in FIG6. A person of ordinary skill in the art will understand that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and appropriate combinations thereof. Some or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on computer-readable media, which may include computer storage media (or non-transitory media) and communication media (or temporary media). As is well known to those of ordinary skill in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and may include any information delivery medium.

Claims

1. A robot arm control method based on deep learning, characterized in that: include: Acquire an ultrasonic image to be detected, wherein the ultrasonic image to be detected represents an image when the robotic arm reaches the body surface of the target object; Inputting the to-be-detected ultrasonic image into a pre-trained deep learning model for image processing to obtain an image recognition result; Determining initial motion trajectory information of the robotic arm; Determine a target motion instruction according to the image recognition result, the initial motion trajectory information and a preset rule; The movement of the robotic arm is controlled according to the target movement instruction.

2. The deep learning-based robotic arm control method according to claim 1, characterized in that: The determining of the target motion instruction according to the image recognition result, the initial motion trajectory information and a preset rule includes: When the image recognition result does not meet the preset conditions, determining the reference force and reference landing point information corresponding to the ultrasonic image to be detected; The target motion instruction is determined according to the reference force, the reference landing point information and the initial motion trajectory information.

3. The deep learning-based robotic arm control method according to claim 2, characterized in that: The initial motion trajectory information includes a target landing point coordinate range, and determining the target motion instruction according to the reference force, the reference landing point information and the initial motion trajectory information includes: When the reference landing point coordinate exceeds the target landing point coordinate range, the target motion instruction is determined according to a reference distance value and reference direction information, wherein the reference distance value is a distance value between the reference landing point coordinate and a boundary point of the target landing point coordinate range, and the reference direction information is a direction vector of the reference landing point coordinate from the nearest boundary point of the target landing point coordinate range; or, When the reference force is less than the target force, the target motion instruction is determined according to the reference force and the target force, and the reference force is the force applied by the robot arm to the body surface of the target object.

4. The deep learning-based robotic arm control method according to claim 1, characterized in that: The deep learning model includes a first convolutional layer, a maximum pooling layer, a second convolutional layer and a fully connected layer connected in sequence, and the ultrasound image to be detected is input into a pre-trained deep learning model for image processing to obtain an image recognition result, including: Inputting the ultrasonic image to be detected into the first convolution layer for convolution operation to obtain a first intermediate image; Inputting the first intermediate image into the maximum pooling layer for sampling processing to obtain a second intermediate image; Inputting the second intermediate image into the second convolutional layer for convolution operation to obtain a third intermediate image; The third intermediate image is input into the second convolutional layer for image processing to obtain the image recognition result.

5. The deep learning-based robotic arm control method according to claim 4, characterized in that: The deep learning model also includes a first BN layer, a second BN layer, a first activation layer, and a second activation layer. The first BN layer and the first activation layer are respectively connected to the first convolutional layer, and the second BN layer and the second activation layer are respectively connected to the first convolutional layer.

6. The deep learning-based robotic arm control method according to claim 2, characterized in that: After controlling the movement of the robot arm according to the target movement instruction, the method further includes: Reacquiring a new ultrasonic image to be detected, inputting the new ultrasonic image to be detected into the deep learning model for image processing, and obtaining a new image recognition result; When the new image recognition result meets the preset condition, the robotic arm is controlled to stop moving.

7. The deep learning-based robotic arm control method according to claim 1, characterized in that: Before inputting the ultrasonic image to be detected into a pre-trained deep learning model for image processing to obtain an image recognition result, the method further includes: Get preset image preprocessing rules; The ultrasonic image to be detected is preprocessed according to the image preprocessing rule to obtain a preprocessed ultrasonic image to be detected.

8. A surgical robot, characterized in that: include: An image acquisition module, the image acquisition module is used to acquire an ultrasonic image to be detected, the ultrasonic image to be detected represents an image when the robot arm reaches the body surface of the target object; An image processing module, wherein the image processing module is used to input the ultrasonic image to be detected into a pre-trained deep learning model for image processing to obtain an image recognition result; An initial motion trajectory information acquisition module, wherein the initial motion trajectory information acquisition module is used to determine the initial motion trajectory information of the robotic arm; A target motion instruction acquisition module, the target motion instruction acquisition module is used to determine the target motion instruction according to the image recognition result, the initial motion trajectory information and a preset rule; A motion control module, wherein the motion control module is used to control the motion of the robotic arm according to the target motion instruction.

9. A surgical robot, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the deep learning-based robotic arm control method as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the deep learning-based robotic arm control method as described in any one of claims 1 to 7.

Images