Method, device and system for orthopedic surgical robot to intelligently perceive anatomical structure

Abstract

The present invention provides a method, device and system for an orthopedic surgical robot to intelligently perceive anatomical structures, the method comprising: after driving a robotic arm trolley to move to a target position away from an operating table and adjusting the robotic arm of the robotic arm trolley to a target angle with the operating table, controlling the camera on the robotic arm trolley to aim at the operating table to take a picture and obtain a human joint image; inputting the human joint image into a trained soft tissue recognition network model to perform a soft tissue feature extraction operation to obtain a preliminary soft tissue feature output by the soft tissue recognition network model; inputting the preliminary soft tissue feature into a trained feature classification network model to perform a classification operation to obtain a soft tissue region output by the feature classification network model. The method, device and system for an orthopedic surgical robot to intelligently perceive anatomical structures provided by the present invention can automatically and accurately identify soft tissues and help surgeons accurately mark tissues and blood vessels.

Description

Technical Field

[0001] The present invention relates to the field of robotics technology, and in particular to a method, device and system for an orthopedic surgical robot to intelligently perceive anatomical structures. Background Art

[0002] In surgical clinics, surgeons need to accurately mark tissues and blood vessels and locate the edges of tissues in order to accurately and completely cut the tissues. This process currently generally relies on the surgeon's subjective clinical experience, which cannot accurately and completely cut the tissues, and may also remove more normal tissues, ultimately causing some iatrogenic damage and failing to protect the rights and interests of patients. Therefore, this paper proposes a method that can automatically and accurately identify soft tissues and help surgeons accurately mark tissues and blood vessels. Summary of the invention

[0003] The present invention provides a method, device and system for an orthopedic surgical robot to intelligently perceive anatomical structure, which can automatically and accurately identify soft tissue and help surgeons accurately mark tissue and blood vessels.

[0004] The present invention provides a method for an orthopedic surgical robot to intelligently sense anatomical structure, comprising:

[0005] After driving the robotic arm trolley to move to a target position away from the operating bed and adjusting the robotic arm of the robotic arm trolley to a target angle with the operating bed, controlling the camera on the robotic arm trolley to aim at the operating bed to take a picture to obtain a human joint image;

[0006] Inputting the human joint image into a trained soft tissue recognition network model to perform a soft tissue feature extraction operation, and obtaining preliminary soft tissue features output by the soft tissue recognition network model;

[0007] The preliminary soft tissue features are input into a trained feature classification network model for classification, and a soft tissue region output by the feature classification network model is obtained.

[0008] According to the method for intelligently perceiving anatomical structures of orthopedic surgical robots provided by the present invention, the soft tissue recognition network model is obtained by training the UNet network model based on preset human joint image samples and their corresponding preliminary soft tissue features;

[0009] The feature classification network model is obtained by training the PointRend network model based on preset soft tissue preliminary feature samples and their corresponding soft tissue areas.

[0010] The method for intelligently sensing anatomical structure of an orthopedic surgical robot provided by the present invention also includes:

[0011] Before driving the robotic arm trolley to move to the target position and adjusting the robotic arm to the target angle, obtaining the posture information between the robotic arm trolley and the operating table;

[0012] Determining the positioning parameters of the robotic arm trolley according to the posture information and the preset posture threshold;

[0013] According to the positioning parameters of the robotic arm trolley, the robotic arm trolley is driven to move to the target position, and the robotic arm is adjusted to the target angle.

[0014] According to the method for intelligently sensing anatomical structure of an orthopedic surgical robot provided by the present invention, the acquisition of the position information between the robotic arm trolley and the operating table includes:

[0015] The position information sent by the navigation trolley is obtained, where the position information is sensed and collected by the navigation trolley on the first reference device on the robotic arm trolley and the second reference device on the operating table.

[0016] According to the method for intelligently sensing anatomical structure of an orthopedic surgical robot provided by the present invention, determining the positioning parameters of the robotic arm trolley according to the posture information and a preset posture threshold value includes:

[0017] Based on the posture information, determine the coordinates of the robot arm trolley and the operating table in the same coordinate system;

[0018] Based on the coordinate difference between the coordinates of the robotic arm trolley and the coordinates of the operating table, and the preset posture threshold, the positioning parameters of the robotic arm trolley are determined.

[0019] The method for orthopedic surgical robot to intelligently sense anatomical structure according to the present invention also includes:

[0020] When the coordinate difference is less than the preset posture threshold, it is determined that the robotic arm trolley is located at the target position and the robotic arm is located at the target angle.

[0021] The method for orthopedic surgical robot to intelligently sense anatomical structure according to the present invention also includes:

[0022] After determining that the robotic arm trolley is located at the target position and the robotic arm is located at the target angle, and before controlling the camera on the robotic arm trolley to aim at the operating table to take a picture, the robotic arm trolley and the robotic arm are locked.

[0023] The present invention also provides a device for orthopedic surgical robot to intelligently sense anatomical structure, comprising:

[0024] An acquisition module is used to control the camera on the mechanical table trolley to aim at the operating table to take pictures and acquire human joint images after driving the mechanical table trolley to move to a target position away from the operating table and adjusting the mechanical arm of the mechanical table trolley to a target angle with the operating table;

[0025] An extraction module, used for inputting the human joint image into a trained soft tissue recognition network model to perform a soft tissue feature extraction operation, and obtaining preliminary soft tissue features output by the soft tissue recognition network model;

[0026] The classification module is used to input the preliminary soft tissue features into a trained feature classification network model for classification operation to obtain the soft tissue area output by the feature classification network model.

[0027] The present invention also provides a system for orthopedic surgical robots to intelligently sense anatomical structures, the system comprising a robotic arm trolley and a main control trolley;

[0028] The main control trolley is communicatively connected with the robotic arm trolley, and is used to execute any of the above-mentioned methods for intelligently sensing anatomical structures of orthopedic surgical robots.

[0029] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method of intelligently sensing anatomical structure of an orthopedic surgical robot as described in any one of the above is implemented.

[0030] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for intelligently sensing anatomical structure of an orthopedic surgical robot as described in any of the above.

[0031] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the above-mentioned methods for intelligently sensing anatomical structures of an orthopedic surgical robot.

[0032] The method, device and system for intelligently sensing anatomical structures of orthopedic surgical robots provided by the present invention control the camera on the robotic arm trolley to take pictures of the operating table to obtain human joint images, and then extract the preliminary features of soft tissue in the human joint images based on the soft tissue recognition network model, and then classify the preliminary features of soft tissue based on the feature classification network model to obtain soft tissue areas. The process from obtaining human joint images to finally extracting soft tissue areas is automated, and soft tissues can be automatically and accurately identified to help surgeons accurately mark tissues and blood vessels. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0034] Figure 1 This is one of the flow charts of the method for the orthopedic surgical robot to intelligently sense anatomical structure provided by the present invention;

[0035] Figure 2 This is a principle block diagram of the orthopedic robot provided by the present invention;

[0036] Figure 3 is a schematic diagram of the structure of the orthopedic robot provided by the present invention;

[0037] Figure 4 It is a schematic diagram of installing a navigation module on a navigation trolley provided by the present invention;

[0038] Figure 5 is a schematic diagram of soft tissue recognition results provided by the present invention;

[0039] Figure 6 It is a schematic diagram of the camera device provided by the present invention acquiring an image of a human joint;

[0040] Figure 7 It is a processing flow chart of human joint images provided by the present invention;

[0041] Figure 8 It is a schematic diagram of the structure of the device for intelligently sensing anatomical structure of an orthopedic surgical robot provided by the present invention;

[0042] Fig. 9 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] Combine the following Figure 1-Figure 9 The present invention describes the method, device and system for orthopedic surgical robots to intelligently perceive anatomical structures.

[0045] like Figure 1As shown, the present invention provides a method for an orthopedic surgical robot to intelligently sense anatomical structure, comprising:

[0046] Step 110: After driving the robotic arm trolley to move to a target position away from the operating bed and adjusting the robotic arm of the robotic arm trolley to a target angle with the operating bed, control the camera on the robotic arm trolley to aim at the operating bed to take a picture to obtain a human joint image.

[0047] It is understandable that if Figure 2 and Figure 3 As shown, the orthopedic robot includes a main control trolley 210 , a mechanical arm trolley 220 and a navigation trolley 230 . The method provided by the present invention can be executed by the main control trolley 210 .

[0048] The main control trolley 210 includes a control processor, input and output communication cables, a power cord and a display. The control processor is provided with algorithm software to execute the method provided by the present invention. The control processor is connected to the robot arm trolley 220 through input and output communication. The main control trolley 210 can display the positions of the robot arm trolley 220 and the operating table 240 through the display.

[0049] The robot arm trolley 220 is provided with a magnetic sensor for autonomous route identification, and an intelligent motor drive and steering mechanism.

[0050] The camera device on the robot arm trolley 220 aims at the operating table 240 to take pictures, and after acquiring the human joint image, the human joint image is sent to the main control trolley 210 .

[0051] Step 120: input the human joint image into a trained soft tissue recognition network model to perform a soft tissue feature extraction operation, and obtain preliminary soft tissue features output by the soft tissue recognition network model.

[0052] It is understandable that the soft tissue recognition network model can be trained on the cloud server first and then sent to the main control vehicle 210, which recognizes the human joint image and extracts the soft tissue features. Alternatively, the training of the soft tissue recognition network model can be completed locally on the main control vehicle 210.

[0053] Step 130: Input the preliminary soft tissue features into the trained feature classification network model for classification operation to obtain the soft tissue area output by the feature classification network model.

[0054] It is understandable that the feature classification network model can be trained on the cloud server and then sent to the main control vehicle 210, or the feature classification network model can be trained locally on the main control vehicle 210.

[0055] In some embodiments, the soft tissue recognition network model is obtained by training a UNet network model based on preset human joint image samples and their corresponding preliminary soft tissue features;

[0056] The feature classification network model is obtained by training the PointRend network model based on preset soft tissue preliminary feature samples and their corresponding soft tissue areas.

[0057] It can be understood that the UNet network model is mainly composed of two parts: feature extraction and feature restoration. The feature extraction part is composed of basic convolution modules and pooling operations. The convolution module includes convolution, batch normalization, activation and other operations; the feature restoration part is mainly completed by convolution modules and upsampling operations. The upsampling method is bilinear interpolation, the purpose is to restore the size of the feature map. The UNet network model can further be an Attention UNet network model.

[0058] PointRendThe PointRend network model is mainly composed of convolution and fully connected layers. The PointRend network model can reclassify the boundaries segmented by the UNet network model to obtain better boundary segmentation effects and can more accurately identify soft tissue areas.

[0059] In some embodiments, the method for an orthopedic surgical robot to intelligently sense anatomical structure further includes:

[0060] Before driving the robot arm trolley 220 to move to the target position and adjusting the robot arm to the target angle, obtaining the posture information between the robot arm trolley 220 and the operating table 240;

[0061] Determine the positioning parameters of the robotic arm trolley 220 according to the posture information and the preset posture threshold;

[0062] According to the positioning parameters of the robotic arm trolley 220 , the robotic arm trolley 220 is driven to move to the target position, and the robotic arm is adjusted to the target angle.

[0063] It can be understood that the posture information between the robotic arm trolley 220 and the operating bed 240 may include the distance between the robotic arm trolley 220 and the operating bed 240, the orientation of the robotic arm trolley 220 relative to the operating bed 240, and the distance between the robotic arm of the robotic arm trolley 220 and the operating bed 240 and the orientation of the robotic arm relative to the operating bed 240.

[0064] The position information is compared with a preset position threshold, the difference between the position information and the preset position threshold is determined, and the positioning parameters of the robot arm trolley 220 are determined.

[0065] Furthermore, the posture threshold includes a first distance threshold, a second distance threshold, a first orientation threshold and a second orientation threshold.

[0066] The distance between the robotic arm trolley 220 and the operating bed 240 is compared with the first distance threshold, and the difference between the two is determined to be the first distance difference; the distance between the robotic arm and the operating bed 240 is compared with the second distance threshold, and the difference between the two is determined to be the second distance difference.

[0067] The orientation of the robotic arm trolley 220 relative to the operating bed 240 is compared with the first orientation threshold, and the orientation difference between the two is determined to be the first orientation difference; the orientation of the robotic arm relative to the operating bed 240 is compared with the second orientation threshold, and the orientation difference between the two is determined to be the second orientation difference.

[0068] A first positioning parameter can be determined based on the first distance difference and the first orientation difference, and the first positioning parameter is used to control the running track of the mechanical trolley. A second positioning parameter can be determined based on the second distance difference and the second orientation difference, and the second positioning parameter is used to control the running track of the mechanical arm.

[0069] The robot arm trolley 220 is driven to move to a target position from the operating bed 240 based on the first positioning parameter. After the robot arm trolley 220 is driven to move to the target position, the robot arm is adjusted to a target angle with respect to the operating bed 240 based on the second positioning parameter adjustment.

[0070] In some embodiments, the method for an orthopedic surgical robot to intelligently perceive anatomical structures, wherein the obtaining of the position information between the robotic arm trolley 220 and the operating table 240 includes:

[0071] The position information sent by the navigation trolley 230 is obtained, where the position information is sensed and collected by the navigation trolley 230 on the first reference device on the robotic arm trolley 220 and the second reference device 250 on the operating table 240 .

[0072] It can be understood that both the robotic arm trolley 220 and the navigation trolley 230 are communicatively connected to the main control trolley 210. Furthermore, the robotic arm trolley 220 and the main control trolley 210 can be connected via cable communication or wireless communication, and the navigation trolley 230 and the main control trolley 210 can be connected via cable communication or wireless communication.

[0073] like Figure 4 As shown, the navigation trolley 230 includes a structural frame 232 and a navigation module 231 . The navigation module 231 is fixed on the structural frame 232 . The navigation module 231 is used to collect the position information between the robot arm trolley 220 and the operating table 240 .

[0074] The navigation trolley 230 collects the position information between the robot arm trolley 220 and the operating table 240 and sends the position information to the main control trolley 210 .

[0075] The navigation trolley 230 is provided with a tracking system, which collects feedback signals of the first reference device and the second reference device 250 , and determines the position information between the robot arm trolley 220 and the operating table 240 based on the feedback signals of the first reference device and the second reference device 250 .

[0076] In some embodiments, both the first reference device and the second reference device 250 are optical feedback components.

[0077] It can be understood that the tracking system provided on the navigation trolley 230 is an optical tracking system, which can feed back the posture information between the robotic arm trolley 220 and the operating bed 240 to the main control trolley 210 through a signal cable, and the main control trolley 210 will perform comparative calculations until the robotic arm trolley 220 and the robotic arm match the preset posture.

[0078] In some embodiments, determining the positioning parameters of the robotic arm trolley 220 according to the posture information and the preset posture threshold includes:

[0079] Based on the posture information, determine the coordinates of the robot arm trolley 220 and the coordinates of the operating table 240 in the same coordinate system;

[0080] Based on the coordinate difference between the coordinates of the robotic arm trolley 220 and the coordinates of the operating table 240 and the preset posture threshold, the positioning parameters of the robotic arm trolley 220 are determined.

[0081] It can be understood that the posture information between the robotic arm trolley 220 and the operating bed 240 can be converted into the robotic arm trolley 220 coordinates and the operating bed 240 coordinates in the robotic arm coordinate system. The coordinate system can be based on the center of the robotic arm as the coordinate origin, the length direction of the robotic arm as the horizontal axis direction, and the direction perpendicular to the length direction of the robotic arm as the vertical axis direction.

[0082] In the robotic arm coordinate system, based on the difference between the horizontal coordinate of the robotic arm trolley 220 and the horizontal coordinate of the operating bed 240, and the first preset posture threshold, the positioning parameters of the robotic arm trolley 220 corresponding to the horizontal axis direction are determined, and based on the positioning parameters corresponding to the horizontal axis direction, the operation of the robotic arm trolley 220 in the horizontal axis direction is controlled so that the difference between the horizontal coordinate of the robotic arm trolley 220 and the horizontal coordinate of the operating bed 240 is less than the first preset posture threshold.

[0083] Based on the difference between the longitudinal coordinate of the robotic arm trolley 220 and the longitudinal coordinate of the operating bed 240, and the second preset posture threshold, the positioning parameters of the robotic arm trolley 220 in the longitudinal axis direction are determined, and based on the positioning parameters corresponding to the longitudinal axis direction, the operation of the robotic arm trolley 220 in the longitudinal axis direction is controlled so that the difference between the longitudinal coordinate of the robotic arm trolley 220 and the longitudinal coordinate of the operating bed 240 is less than the second preset posture threshold.

[0084] In some embodiments, the method for an orthopedic surgical robot to intelligently sense anatomical structure further includes:

[0085] When the coordinate difference is less than the preset posture threshold, it is determined that the robotic arm trolley 220 is located at the target position and the robotic arm is located at the target angle.

[0086] It can be understood that when the difference between the horizontal coordinate of the robotic arm trolley 220 and the horizontal coordinate of the operating bed 240 is less than the first preset posture threshold, and the difference between the vertical coordinate of the robotic arm trolley 220 and the vertical coordinate of the operating bed 240 is less than the second preset posture threshold, it can be determined that the robotic arm trolley 220 is at the target position and the robotic arm is at the target angle.

[0087] In some embodiments, the method for an orthopedic surgical robot to intelligently sense anatomical structure further includes:

[0088] After determining that the robotic arm trolley 220 is located at the target position and the robotic arm is located at the target angle, and before controlling the camera on the robotic arm trolley to aim at the operating table 240 to take a picture, the robotic arm trolley 220 and the robotic arm are locked.

[0089] It can be understood that after determining that the robotic arm trolley 220 is located at the target position and the robotic arm is at the target angle, the robotic arm trolley 220 and the robotic arm are locked, that is, the robotic arm trolley 220 and the robotic arm are fixed to prevent the robotic arm from shaking during subsequent work.

[0090] The schematic diagram of the soft tissue recognition result obtained by the method provided by the present invention is as follows: Figure 5 As shown, a soft tissue area 510 and a bone area 520 are included.

[0091] In other embodiments, Figure 6 As shown, the camera device 221 takes a picture of the leg area 610 of the patient on the operating table 240 to obtain a human joint image. The leg area 610 is cut open with a scalpel to expose bones and muscle tissues.

[0092] In other embodiments, after obtaining the human joint image, Figure 7The UNet network model and PointRend network model shown are processed. Figure 7 The UNet network model shown contains three convolutional layers of 258x128x128, 128x64x128, and 64x32x256. The human joint image is roughly segmented through the UNet network model to obtain the preliminary features of the soft tissue, and then the PointRend network model is used to perform pixel-level segmentation to obtain the soft tissue area.

[0093] In summary, the method for intelligent perception of anatomical structure by an orthopedic surgical robot provided by the present invention includes: after driving the robotic arm trolley 220 to move to a target position away from an operating bed 240 and adjusting the robotic arm of the robotic arm trolley 220 to a target angle with the operating bed 240, controlling the camera on the robotic arm trolley 220 to aim at the operating bed 240 to take a picture and obtain a human joint image; inputting the human joint image into a trained soft tissue recognition network model to perform a soft tissue feature extraction operation to obtain preliminary soft tissue features output by the soft tissue recognition network model; inputting the preliminary soft tissue features into a trained feature classification network model to perform a classification operation to obtain a soft tissue area output by the feature classification network model.

[0094] In the method for orthopedic surgical robot to intelligently perceive anatomical structure provided by the present invention, the camera on the robot arm trolley 220 is controlled to take a picture of the operating table 240 to obtain a human joint image, and then the preliminary features of the soft tissue in the human joint image are extracted based on the soft tissue recognition network model, and then the preliminary features of the soft tissue are feature classified based on the feature classification network model to obtain the soft tissue area. The process from obtaining the human joint image to the final extraction of the soft tissue area is automated, and the soft tissue can be automatically and accurately identified to help surgeons accurately mark tissue blood vessels.

[0095] The device for intelligently sensing anatomical structure of an orthopedic surgical robot provided by the present invention is described below. The device for intelligently sensing anatomical structure of an orthopedic surgical robot described below and the method for intelligently sensing anatomical structure of an orthopedic surgical robot described above can be referenced to each other.

[0096] like Figure 8 As shown, the present invention also provides a device 800 for orthopedic surgical robot to intelligently sense anatomical structure, comprising:

[0097] The acquisition module 810 is used to control the camera on the mechanical table trolley to aim at the operating bed 240 to take a picture, and acquire the human body joint image, after driving the mechanical table trolley 220 to move to the target position from the operating bed 240 and adjusting the mechanical arm of the mechanical table trolley 220 to the target angle with the operating bed 240;

[0098] An extraction module 820 is used to input the human joint image into a trained soft tissue recognition network model to perform a soft tissue feature extraction operation, and obtain preliminary soft tissue features output by the soft tissue recognition network model;

[0099] The classification module 830 is used to input the preliminary soft tissue features into the trained feature classification network model for classification operation to obtain the soft tissue area output by the feature classification network model.

[0100] The present invention also provides a system for orthopedic surgical robots to intelligently sense anatomical structures, the system comprising a robotic arm trolley 220 and a main control trolley 210 .

[0101] The main control trolley 210 is communicatively connected with the robotic arm trolley 220, and is used to execute any of the above-mentioned methods for intelligently sensing anatomical structures of orthopedic surgical robots.

[0102] The electronic device, computer program product and storage medium provided by the present invention are described below. The electronic device, computer program product and storage medium described below can be referenced to each other with the method for intelligent perception of anatomical structure by an orthopedic surgical robot described above.

[0103] Fig. 9 An example of a physical structure diagram of an electronic device is shown in FIG. Fig. 9 As shown, the electronic device may include: a processor 910, a communication interface 920, a memory 930 and a communication bus 940, wherein the processor 910, the communication interface 920 and the memory 930 communicate with each other through the communication bus 940. The processor 910 may call the logic instructions in the memory 930 to execute the method for the orthopedic surgical robot to intelligently perceive the anatomical structure, the method comprising:

[0104] After driving the robotic arm trolley to move to a target position away from the operating bed and adjusting the robotic arm of the robotic arm trolley to a target angle with the operating bed, controlling the camera on the robotic arm trolley to aim at the operating bed to take a picture to obtain a human joint image;

[0105] Inputting the human joint image into a trained soft tissue recognition network model to perform a soft tissue feature extraction operation, thereby obtaining preliminary soft tissue features output by the soft tissue recognition network model;

[0106] The preliminary soft tissue features are input into a trained feature classification network model for classification, and a soft tissue region output by the feature classification network model is obtained.

[0107] In addition, the logic instructions in the above-mentioned memory 930 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0108] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the method of intelligently sensing anatomical structure of an orthopedic surgical robot provided by the above methods, the method comprising:

[0109] After driving the robotic arm trolley to move to a target position away from the operating bed and adjusting the robotic arm of the robotic arm trolley to a target angle with the operating bed, controlling the camera on the robotic arm trolley to aim at the operating bed to take a picture to obtain a human joint image;

[0110] Inputting the human joint image into a trained soft tissue recognition network model to perform a soft tissue feature extraction operation, thereby obtaining preliminary soft tissue features output by the soft tissue recognition network model;

[0111] The preliminary soft tissue features are input into a trained feature classification network model for classification, and a soft tissue region output by the feature classification network model is obtained.

[0112] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the method for an orthopedic surgical robot to intelligently sense anatomical structure provided by the above methods, the method comprising:

[0113] After driving the robotic arm trolley to move to a target position away from the operating bed and adjusting the robotic arm of the robotic arm trolley to a target angle with the operating bed, controlling the camera on the robotic arm trolley to aim at the operating bed to take a picture to obtain a human joint image;

[0114] Inputting the human joint image into a trained soft tissue recognition network model to perform a soft tissue feature extraction operation, thereby obtaining preliminary soft tissue features output by the soft tissue recognition network model;

[0115] The preliminary soft tissue features are input into a trained feature classification network model for classification, and a soft tissue region output by the feature classification network model is obtained.

[0116] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0117] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for orthopedic surgical robot to intelligently perceive anatomical structure, characterized in that: include: After driving the robotic arm trolley to move to a target position away from the operating bed and adjusting the robotic arm of the robotic arm trolley to a target angle with the operating bed, controlling the camera on the robotic arm trolley to aim at the operating bed to take a picture to obtain a human joint image; Inputting the human joint image into a trained soft tissue recognition network model to perform a soft tissue feature extraction operation, thereby obtaining preliminary soft tissue features output by the soft tissue recognition network model; Inputting the preliminary soft tissue features into a trained feature classification network model for classification, and obtaining a soft tissue region output by the feature classification network model; Before driving the robotic arm trolley to move to a target position from the operating bed and adjusting the robotic arm of the robotic arm trolley to a target angle with respect to the operating bed, the method further includes: Acquiring the position information between the robotic arm trolley and the operating table; Determining the positioning parameters of the robotic arm trolley according to the posture information and the preset posture threshold; According to the positioning parameters of the robotic arm trolley, drive the robotic arm trolley to move to the target position, and adjust the robotic arm to the target angle; The preset posture threshold comprises a first preset posture threshold and a second preset posture threshold; and determining the positioning parameters of the robotic arm trolley according to the posture information and the preset posture threshold comprises: Based on the posture information, determine the coordinates of the robot arm trolley and the operating table in the same coordinate system; Based on the difference between the horizontal coordinate of the robotic arm trolley and the horizontal coordinate of the operating table, and the first preset posture threshold, determining the positioning parameter of the robotic arm trolley corresponding to the horizontal axis direction; Based on the difference between the longitudinal coordinate of the robotic arm trolley and the longitudinal coordinate of the operating table, and the second preset posture threshold, the positioning parameters of the robotic arm trolley corresponding to the longitudinal axis direction are determined.

2. The method for intelligently sensing anatomical structure of an orthopedic surgical robot according to claim 1, characterized in that: The soft tissue recognition network model is obtained by training the UNet network model based on preset human joint image samples and their corresponding preliminary soft tissue features; The feature classification network model is obtained by training the PointRend network model based on preset soft tissue preliminary feature samples and their corresponding soft tissue areas.

3. The method for intelligently sensing anatomical structure of an orthopedic surgical robot according to claim 1, characterized in that: The obtaining of the position information between the robotic arm trolley and the operating table includes: The position information sent by the navigation trolley is obtained, where the position information is sensed and collected by the navigation trolley on the first reference device on the robotic arm trolley and the second reference device on the operating table.

4. The method for intelligently sensing anatomical structure of an orthopedic surgical robot according to claim 1, characterized in that: Also includes: When the coordinate difference is less than the preset posture threshold, it is determined that the robotic arm trolley is located at the target position and the robotic arm is located at the target angle.

5. The method for intelligently sensing anatomical structure of an orthopedic surgical robot according to any one of claims 1 to 4, characterized in that: Also includes: After determining that the robotic arm trolley is located at the target position and the robotic arm is located at the target angle, and before controlling the camera on the robotic arm trolley to aim at the operating table to take a picture, the robotic arm trolley and the robotic arm are locked.

6. A device for orthopedic surgical robots to intelligently sense anatomical structures, characterized in that: include: an acquisition module, for controlling a camera on the robotic arm trolley to aim at the operating bed and take a picture to acquire a human joint image after driving the robotic arm trolley to move to a target position from the operating bed and adjusting the robotic arm of the robotic arm trolley to a target angle with the operating bed; An extraction module, used for inputting the human joint image into a trained soft tissue recognition network model to perform a soft tissue feature extraction operation, and obtaining preliminary soft tissue features output by the soft tissue recognition network model; A classification module, used for inputting the preliminary soft tissue features into a trained feature classification network model for classification operation to obtain the soft tissue region output by the feature classification network model; The device for intelligently sensing anatomical structure of the orthopedic surgical robot also includes: A posture information acquisition module, used to acquire the posture information between the robotic arm trolley and the operating table; A positioning parameter determination module, used to determine the positioning parameters of the robotic arm trolley according to the posture information and a preset posture threshold; An adjustment module, used for driving the robotic arm trolley to move to the target position and adjusting the robotic arm to the target angle according to the positioning parameters of the robotic arm trolley; The preset posture threshold comprises a first preset posture threshold and a second preset posture threshold; and determining the positioning parameters of the robotic arm trolley according to the posture information and the preset posture threshold comprises: Based on the posture information, determine the coordinates of the robot arm trolley and the operating table in the same coordinate system; Based on the difference between the horizontal coordinate of the robotic arm trolley and the horizontal coordinate of the operating table, and the first preset posture threshold, determining the positioning parameter of the robotic arm trolley corresponding to the horizontal axis direction; Based on the difference between the longitudinal coordinate of the robotic arm trolley and the longitudinal coordinate of the operating table, and the second preset posture threshold, the positioning parameters of the robotic arm trolley corresponding to the longitudinal axis direction are determined.

7. A system for orthopedic surgical robots to intelligently perceive anatomical structures, characterized in that: The system includes a robotic arm trolley and a main control trolley; The main control trolley is communicatively connected with the robotic arm trolley, and is used to execute the method of intelligent perception of anatomical structure by an orthopedic surgical robot as described in any one of claims 1-5.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, it implements the method for intelligently sensing anatomical structure of an orthopedic surgical robot as described in any one of claims 1 to 5.

Images