Human-machine collaborative surgery training interaction method, system and surgical robot

Abstract

The present application is applicable to the field of medical technology, and in particular, relates to a human-machine collaborative surgery training interaction method, system, and surgical robot, the method comprising: obtaining training project information; wherein the training project information includes the training project name, training model, and training target information; obtaining postoperative information based on the training project information; wherein the postoperative information includes operation process information and postoperative model; obtaining a training analysis report based on the postoperative information and the training project information; wherein the training analysis report includes the operation process information and surgical result information. The human-machine collaborative surgery training interaction method, system, and surgical robot provided in the embodiments of the present application can improve the problem of doctors' lack of proficiency in operating the surgical arm of the surgical robot.

Description

Technical Field

[0001] The present application belongs to the field of medical technology, and in particular, relates to a human-machine collaborative surgery training interaction method, system and surgical robot. Background Art

[0002] Human-machine collaborative surgery refers to a surgical method in which surgical robots and human doctors cooperate with each other to complete surgical tasks. In this mode, surgical robots, as auxiliary equipment, use their high precision, high stability and special capabilities, such as fine operation and stable control, to help human doctors better complete the operation. At the same time, human doctors use their professional knowledge and skills to control and make decisions on the overall surgical process to ensure the safety and success of the operation.

[0003] However, the operation of surgical robots is difficult and depends on the operator's level. Unskilled doctors may collide or rub against the patient's organs when operating the surgical arm during surgery, causing secondary injuries to the patient. Summary of the invention

[0004] The embodiments of the present application provide a human-machine collaborative surgery training interaction method, system and surgical robot, which can improve the problem of doctors' lack of proficiency in operating the surgical arm of the surgical robot.

[0005] In a first aspect, an embodiment of the present application provides a human-machine collaborative surgery training interaction method, comprising:

[0006] Acquire training project information; wherein the training project information includes a training project name, a training model and training target information, the training model is a three-dimensional digital model reflecting the patient's body structure corresponding to the training project name, and the training target information reflects the three-dimensional digital model of the patient's body structure after surgery;

[0007] Obtaining postoperative information based on the training project information; wherein the postoperative information includes operation process information and a postoperative model, the operation process information includes information reflecting the operation instructions input by the user, and the postoperative model is the training model after the operation;

[0008] A training analysis report is obtained based on the postoperative information and the training project information; wherein the training analysis report includes the operation process information and the surgical result information, and the surgical result information reflects the difference between the three-dimensional digital model reflected by the training target information and the three-dimensional digital model reflected by the postoperative model.

[0009] The above technical solutions in the embodiments of the present application have at least the following technical effects:

[0010] The human-machine collaborative surgery training interaction method provided in the embodiment of the present application first obtains the training project information including the training project name, training model and training target information to determine the project that the user needs to train and the goal that needs to be achieved. Then, based on the training project information, postoperative information including operation process information and postoperative model is obtained to provide reference and data support for evaluating the user's proficiency and analyzing the problems encountered by the user during the training process. Then, based on the postoperative information and the training project information, a training analysis report including the operation process information and surgical result information is obtained to intuitively display the results of the user's training and the problems encountered by the user during the training process, thereby improving the user's operation proficiency and improving the doctor's lack of proficiency in operating the surgical arm of the surgical robot.

[0011] In a possible implementation of the first aspect, the three-dimensional digital model reflected by the training model includes at least two first positioning markers and at least one first area, and the three-dimensional digital model reflected by the training target information includes at least two second positioning markers, and the first positioning marker corresponds to one second positioning marker. When all of the first positioning markers coincide with the corresponding second positioning markers, the three-dimensional digital model reflected by the training model can partially coincide with the three-dimensional digital model reflected by the training target information.

[0012] In a possible implementation manner of the first aspect, obtaining postoperative information based on the training item information includes:

[0013] Acquire a robotic arm model; wherein the robotic arm model is a three-dimensional digital model reflecting the part of the surgical robot extending into the patient's body, the robotic arm model includes a moving part and an operating part, the moving part reflects the area range of the part of the robotic arm model used to execute the moving instruction, the moving instruction reflects the moving direction and moving speed of the robotic arm model, the operating part reflects the area range of the part of the robotic arm model used to execute the surgical operation instruction, the surgical operation instruction is used to instruct the operating part of the robotic arm model to perform a grasping or cutting action, and the three-dimensional digital model reflected by the robotic arm model does not contact the three-dimensional digital model reflected by the training model in the default position;

[0014] Obtaining an operation instruction; wherein the operation instruction includes the movement instruction or the surgical operation instruction;

[0015] The postoperative information is obtained based on the robotic arm model, the operation instructions and the training model.

[0016] In a possible implementation manner of the first aspect, obtaining the postoperative information based on the robotic arm model, the operation instruction, and the training model includes:

[0017] Determine the instruction type of the operation instruction, and if the operation instruction is a movement instruction, obtain first operation information based on a movement detection method; wherein the first operation information includes the operation instruction reflected by the user input and the position information of the robot arm model;

[0018] If the operation instruction is a surgical operation instruction, second operation information is obtained based on an operation detection method; wherein the second operation information includes the operation instruction reflected by the user input and the position information of the robot arm model;

[0019] confirming the first operation information and the second operation information as the operation process information;

[0020] receiving an end signal, and confirming the training model as the postoperative model; wherein the end signal reflects that the surgical training is completed;

[0021] The operation process information and the postoperative model are confirmed as the postoperative information.

[0022] In a possible implementation manner of the first aspect, obtaining the first operation information based on the motion detection method includes:

[0023] Acquire moving direction information; wherein the moving direction information reflects the moving direction of the robotic arm model;

[0024] Acquire movement speed information; wherein the movement speed information reflects the movement speed of the mechanical arm model;

[0025] Acquire the position information of the robotic arm; wherein the position information of the robotic arm reflects the relative position information of the robotic arm model and the training model;

[0026] The first operation information is obtained based on the robot arm position information, the moving speed information and the moving direction information.

[0027] In a possible implementation manner of the first aspect, obtaining the first operation information based on the robot arm position information, the moving speed information, and the moving direction information includes:

[0028] Determining whether the position of the robotic arm model reflected by the robotic arm position information is within the first area;

[0029] If the position of the robotic arm model reflected by the robotic arm position information is within the first area, and the moving speed reflected by the moving speed information is greater than a preset moving speed, the robotic arm position information and the moving speed information are confirmed as the first operation information;

[0030] If the position of the robotic arm model reflected by the robotic arm position information is not within the first area, the speed reflected by the moving speed information is multiplied by m seconds to obtain the moving distance information, and it is determined whether the robotic arm model is within the first area after moving the distance reflected by the moving distance information in the direction reflected by the moving direction information; wherein the moving distance information reflects a distance, m is a positive number, m=0.5;

[0031] If the robot arm model is located within the first area after moving the distance reflected by the moving distance information in the direction reflected by the moving direction information, the robot arm position information, the moving direction information and the moving speed information are confirmed as the first operation information.

[0032] In a possible implementation manner of the first aspect, obtaining the second operation information based on the operation detection method includes:

[0033] Acquire the position information of the robotic arm; wherein the position information of the robotic arm reflects the relative position information of the robotic arm model and the training model;

[0034] Determining whether the operating part of the robot arm model in the robot arm position information is located in the first area;

[0035] If the operating part of the robot arm model in the robot arm position information is located in the first area, the robot arm position information and the operation instruction are confirmed as the second operation information.

[0036] In a possible implementation manner of the first aspect, obtaining a training analysis report based on the postoperative information and the training item information includes:

[0037] Overlapping the first positioning mark in the three-dimensional digital model reflected by the postoperative model with the corresponding second positioning mark in the three-dimensional digital model reflected by the training target information;

[0038] confirming the volume of the non-overlapping portion of the three-dimensional digital model reflected by the postoperative model and the three-dimensional digital model reflected by the training target information as a non-overlapping volume;

[0039] confirming the volume of the overlapping portion of the three-dimensional digital model reflected by the postoperative model and the three-dimensional digital model reflected by the training target information as the overlapping volume;

[0040] confirming the ratio of the non-overlapping volume to the overlapping volume as the surgery result information;

[0041] The operation result information and the operation process information are confirmed as the training analysis report.

[0042] In a second aspect, an embodiment of the present application provides a human-machine collaborative surgery training interactive system, including:

[0043] An acquisition unit is used to acquire training project information; wherein the training project information includes a training project name, a training model and training target information, wherein the training model is a three-dimensional digital model reflecting the patient's body structure corresponding to the training project name, and the training target information reflects the three-dimensional digital model of the patient's body structure after surgery;

[0044] A first analysis unit is configured to obtain postoperative information based on the training item information; wherein the postoperative information includes operation process information and a postoperative model, the operation process information includes information reflecting the operation instructions input by the user, and the postoperative model is the training model after the operation;

[0045] A second analysis unit is used to obtain a training analysis report based on the postoperative information and the training project information; wherein the training analysis report includes the operation process information and the surgical result information, and the surgical result information includes the difference between the three-dimensional digital model reflected by the training target information and the three-dimensional digital model reflected by the postoperative model.

[0046] In a third aspect, an embodiment of the present application provides a surgical robot comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the methods described in the first aspect when executing the computer program.

[0047] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of the above-mentioned first aspects is implemented.

[0048] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a surgical robot, enables the surgical robot to execute the human-machine collaborative surgery training interaction method described in any one of the first aspects above.

[0049] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 It is a flowchart of a human-machine collaborative surgery training interaction method provided by an embodiment of the present application;

[0052] Figure 2 It is a flowchart of step S200 in the human-machine collaborative surgery training interaction method provided in one embodiment of the present application;

[0053] Figure 3 It is a flowchart of step S231 in the human-machine collaborative surgery training interaction method provided in one embodiment of the present application;

[0054] Figure 4 It is a flowchart of step S232 in the human-machine collaborative surgery training interaction method provided in one embodiment of the present application;

[0055] Figure 5 It is a flowchart of step S300 in the human-machine collaborative surgery training interaction method provided in one embodiment of the present application;

[0056] Figure 6 is a structural schematic diagram of a human-machine collaborative surgery training interactive system provided by an embodiment of the present application;

[0057] Figure 7 It is a schematic diagram of the structure of a surgical robot provided in one embodiment of the present application. DETAILED DESCRIPTION

[0058] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0059] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0060] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0061] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0062] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0063] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0064] In related technologies, human-machine collaborative surgery refers to a surgical method in which surgical robots and human doctors cooperate with each other to complete surgical tasks. In this mode, surgical robots, as auxiliary equipment, use their high precision, high stability and special capabilities, such as fine operation and stable control, to help human doctors complete the operation better. At the same time, human doctors use their professional knowledge and skills to control and make decisions on the entire surgical process to ensure the safety and success of the operation.

[0065] However, the operation of surgical robots is difficult and depends on the operator's level. Unskilled doctors may collide or rub against the patient's organs when operating the surgical arm during surgery, causing secondary injuries to the patient.

[0066] To solve the above problems, the embodiments of the present application provide a human-machine collaborative surgical training interaction method, system and surgical robot. In this method, by first obtaining training project information including training project name, training model and training target information, the project that the user needs to train and the goal that needs to be achieved are determined. Then, based on the training project information, postoperative information including operation process information and postoperative model is obtained to provide reference and data support for evaluating the user's proficiency and analyzing the problems encountered by the user during the training process. Based on the postoperative information and training project information, a training analysis report including operation process information and surgical result information is obtained, which intuitively displays the results of the user's training and the problems encountered by the user during the training process, improves the user's operation proficiency, and improves the doctor's lack of proficiency in operating the surgical arm of the surgical robot.

[0067] The human-machine collaborative surgery training interaction method provided in the embodiment of the present application can be applied to a surgical robot. In this case, the surgical robot is the executor of the human-machine collaborative surgery training interaction method provided in the embodiment of the present application. The embodiment of the present application does not impose any restrictions on the specific type of the surgical robot.

[0068] For example, the surgical robot may include a control device, a display device, and an input device. The control device is respectively connected to the display device and the input device in communication. For example, the control device may be a computing device such as a laptop computer or a cloud server, but is not limited thereto. The display device is a device that can directly display the patient's physical condition or training model to the user. For example, the display device may be a medical grayscale display, an OLED liquid crystal display, etc., but is not limited thereto. The input device is a device that can obtain instructions issued by the user and send the instructions to the control device. For example, the input device may be a master hand controller, a voice control device, etc., but is not limited thereto.

[0069] In order to better understand the human-machine collaborative surgery training interaction method provided in the embodiment of the present application, the specific implementation process of the human-machine collaborative surgery training interaction method provided in the embodiment of the present application is exemplarily introduced below.

[0070] Figure 1 A schematic flow chart of a human-machine collaborative surgery training interaction method provided in an embodiment of the present application is shown. The human-machine collaborative surgery training interaction method includes:

[0071] S100, obtaining training project information; wherein the training project information includes a training project name, a training model and training target information, the training model is a three-dimensional digital model corresponding to the training project name reflecting the patient's body structure, and the training target information reflects a three-dimensional digital model of the patient's body structure after surgery.

[0072] It can be understood that the method of obtaining the training project information can be to receive the training project name input by the user, and then search the training project name in the preset database to obtain the training model and training target information corresponding to the training project name, or to receive data transmitted by the user, etc., but not limited to this. Among them, the preset database is a database that includes all known training project names and training models corresponding to the training project names. The preset database can be an object database, a relational database, etc., but not limited to this. Obtaining the training project information including the training project name, training model and training target information can determine the project that the user needs to train and the goal to be achieved, and provide a basis for subsequent steps.

[0073] In one possible implementation, the three-dimensional digital model reflected by the training model includes at least two first positioning markers and at least one first area, and the three-dimensional digital model reflected by the training target information includes at least two second positioning markers, and a first positioning marker corresponds to one second positioning marker. When all the first positioning markers coincide with the corresponding second positioning markers, the three-dimensional digital model reflected by the training model can partially coincide with the three-dimensional digital model reflected by the training target information.

[0074] It can be understood that the three-dimensional digital model reflected by the training model can be an interactive physical simulation model, a virtual reality (VR) and an augmented reality (AR) simulation model, etc., but not limited thereto. The three-dimensional digital model reflected by the training target information can be an interactive physical simulation model, a virtual reality (VR) and an augmented reality (AR) simulation model, etc., but not limited thereto. By setting a plurality of one-to-one corresponding positioning marks on the three-dimensional digital model reflected by the training model and the three-dimensional digital model reflected by the training target information, support can be provided for judging the completion of the user's simulation training.

[0075] S200, obtaining postoperative information based on the training project information; wherein the postoperative information includes operation process information and a postoperative model, the operation process information includes information reflecting the operation instructions input by the user, and the postoperative model is a training model after the operation.

[0076] It is understandable that the method of obtaining postoperative information based on the training project information may be to receive the user's changes to the training model in the training project information, and then compare the changed training model with the model reflected by the training target information in the training project information to obtain the postoperative information, or to send the changed training model and the model reflected by the training target information in the training project information to the user, and then receive the data sent back by the user and confirm the data sent back by the user as postoperative information, etc., but is not limited to this. Obtaining postoperative information including operation process information and postoperative models based on the training project information can provide reference and data support for evaluating the user's proficiency and analyzing the problems that the user has in the training process.

[0077] In one possible implementation, see Figure 2 S200, obtaining postoperative information based on the training project information, including:

[0078] S210, obtaining a robotic arm model; wherein the robotic arm model is a three-dimensional digital model reflecting the part of the surgical robot extending into the patient's body, the robotic arm model includes two parts: a moving part and an operating part, the moving part reflects the area range of the part of the robotic arm model used to execute movement instructions, the movement instructions reflect the moving direction and movement speed of the robotic arm model, the operating part reflects the area range of the part of the robotic arm model used to execute surgical operation instructions, the surgical operation instructions are used to instruct the operating part of the robotic arm model to perform grasping or cutting actions, and the three-dimensional digital model reflected by the robotic arm model does not contact the three-dimensional digital model reflected by the training model when in the default position.

[0079] It can be understood that the default position is the position of the robotic arm model at the beginning of surgical training. The way to obtain the robotic arm model can be to receive model data transmitted by the user, or to select a surgical robot corresponding to the training project name according to the training project name, and confirm the part of the surgical robot corresponding to the training project name that extends into the patient's body as the robotic arm model, etc., but not limited to this. The robotic arm model can move according to movement instructions, and can also perform grasping or cutting actions according to surgical operation instructions. Obtaining a robotic arm model that reflects the three-dimensional digital model of the part of the surgical robot extending into the patient's body can provide a basis for subsequent steps.

[0080] S220, obtaining an operation instruction; wherein the operation instruction includes a movement instruction or a surgical operation instruction.

[0081] It is understood that the method of obtaining the operation instruction may be to directly receive the instruction code input by the user, or to receive the data transmitted by the input device of the surgical robot, but is not limited thereto. Obtaining the operation instruction including the movement instruction and the surgical operation instruction can provide a basis for the subsequent steps.

[0082] S230, obtaining postoperative information based on the robotic arm model, operation instructions and training model.

[0083] It can be understood that the postoperative information can be obtained based on the robot model, the operation instruction and the training model by analyzing the movement route of the robot model according to the operation instruction and judging the distance between each path point on the movement route of the robot model and the first area in the training model, or judging whether the robot model will change the state structure of the training model in the first area according to the operation instruction, but it is not limited thereto. Obtaining postoperative information based on the robot model, the operation instruction and the training model can provide a basis for subsequent steps.

[0084] In one possible implementation, see Figure 2, S230, obtaining postoperative information based on the robot arm model, operation instructions and training model, including:

[0085] S231, determining the instruction type of the operation instruction, if the operation instruction is a movement instruction, obtaining first operation information based on a movement detection method; wherein the first operation information includes the operation instruction reflecting the user input and the position information of the robot arm model.

[0086] It can be understood that the first operation information can be obtained based on the mobile detection method by analyzing the movement route of the robot model according to the operation instruction and determining the distance between each path point on the movement route of the robot model and the first area in the training model, or by analyzing the movement speed of the robot model in the first area and the movement speed of the robot model when not in the first area, etc., but is not limited thereto. The first operation information including the operation instruction reflecting the user input and the position information of the robot model obtained based on the mobile detection method can provide a reference for the user to analyze problems encountered during the operation and provide a basis for subsequent steps.

[0087] In one possible implementation, see Figure 3 In step S231, first operation information is obtained based on the motion detection method, including:

[0088] S2311, obtaining moving direction information; wherein the moving direction information reflects the moving direction of the robotic arm model.

[0089] It is understood that the way to obtain the moving direction information may be to receive data transmitted by the user or to receive direction data sent by an input device, but is not limited thereto. Obtaining the moving direction information reflecting the moving direction of the robot arm model can provide a basis for subsequent steps.

[0090] S2312, obtaining movement speed information; wherein the movement speed information reflects the movement speed of the robot arm model.

[0091] It is understood that the way to obtain the moving speed information may be to receive data transmitted by the user or to receive speed data sent by an input device, but is not limited thereto. Obtaining the moving speed information reflecting the moving speed of the robot arm model can provide a basis for subsequent steps.

[0092] S2313, obtaining the robot arm position information; wherein the robot arm position information reflects the relative position information between the robot arm model and the training model.

[0093] It is understandable that the method of obtaining the position information of the robot arm may be to receive data transmitted by the user or to receive position data sent by an input device, but is not limited thereto. Obtaining the position information of the robot arm reflecting the relative position information of the robot arm model and the training model can provide a basis for subsequent steps.

[0094] S2314, obtaining first operation information based on the robot arm position information, moving speed information and moving direction information.

[0095] It can be understood that the first operation information can be obtained based on the robot position information, moving speed information and moving direction information by analyzing the moving route of the robot model according to the moving direction information and the robot position information, and determining the distance between each path point on the moving route of the robot model and the first area in the training model, or by determining the moving speed of the robot model when it is in the first area and when it is not in the first area according to the robot position information and the moving speed information, etc., but is not limited thereto. Obtaining the first operation information based on the robot position information, moving speed information and moving direction information can provide a basis for subsequent steps.

[0096] In one possible implementation, see Figure 3 , S2314, obtaining first operation information based on the robot arm position information, moving speed information and moving direction information, including:

[0097] S23141, determining whether the position of the robotic arm model reflected by the robotic arm position information is within the first area.

[0098] It can be understood that the method for determining whether the position of the mechanical arm model reflected by the mechanical arm position information is within the first area can be to determine whether the position coordinates of the mechanical arm model reflected by the mechanical arm position information are within the position range coordinates reflected by the first area, or to determine the distance between the position of the mechanical arm model reflected by the mechanical arm position information and any point in the first area, etc., but is not limited thereto. Determining whether the position of the mechanical arm model reflected by the mechanical arm position information is within the first area can provide a basis for subsequent steps.

[0099] S23142: If the position of the robot arm model reflected by the robot arm position information is within the first area, and the moving speed reflected by the moving speed information is greater than the preset moving speed, the robot arm position information and the moving speed information are confirmed as the first operation information.

[0100] It is understandable that the preset moving speed can be 1 cm per second, or 2 cm per second, etc., but is not limited to this. If the position of the robotic arm model reflected by the robotic arm position information is within the first area, and the moving speed reflected by the moving speed information is greater than the preset moving speed, it means that the user moves the robotic arm model too fast, which may come into contact with important organs or parts and cause damage. Confirming the robotic arm position information and moving speed information as the first operation information can provide data support for the user to analyze problems encountered during the operation and provide a basis for subsequent steps.

[0101] S23143, if the position of the robotic arm model reflected by the robotic arm position information is not within the first area, multiply the speed reflected by the moving speed information by m seconds to obtain the moving distance information, and determine whether the robotic arm model is within the first area after moving the distance reflected by the moving distance information in the direction reflected by the moving direction information; wherein the moving distance information reflects a distance, m is a positive number, m=0.5.

[0102] It can be understood that the method for determining whether the robotic arm model is located in the first area after moving the distance reflected by the moving distance information in the direction reflected by the moving direction information can be to determine whether the coordinates obtained by adding the coordinates of any point on the robotic arm model to the distance value reflected by the moving distance information are within the position range coordinates reflected by the first area, or to obtain multiple projection views of the robotic arm model and the first area along the same direction, and determine whether the projection of the robotic arm model on the multiple projection views and the projection of the first area are in contact, etc., but not limited to this. Determining whether the robotic arm model is located in the first area after moving the distance reflected by the moving distance information in the direction reflected by the moving direction information can provide a basis for subsequent steps.

[0103] For example, assuming that the speed reflected by the moving speed information is 1 cm per second, m=0.5 seconds, then the value reflected by the moving distance information=1*0.5=0.5 cm.

[0104] S23144, if the robot arm model is located within the first area after moving the distance reflected by the moving distance information in the direction reflected by the moving direction information, the robot arm position information, moving direction information and moving speed information are confirmed as the first operation information.

[0105] It can be understood that if the robot arm model moves in the direction reflected by the moving distance information and is located in the first area, it means that the robot arm model will come into contact with the first area within a time shorter than the user's reaction time, which may cause harm to the patient. Confirming the robot arm position information, moving direction information, and moving speed information as the first operation information can provide data support for the user to analyze problems encountered during the operation and provide a basis for subsequent steps.

[0106] S232, if the operation instruction is a surgical operation instruction, obtain second operation information based on the operation detection method; wherein the second operation information includes the operation instruction reflecting the user input and the position information of the robot arm model.

[0107] It can be understood that the second operation information can be obtained based on the operation detection method by determining whether the operation part of the robot model is within the first area, or by determining the distance between any point on the operation part of the robot model and the first area, but is not limited thereto. The second operation information including the operation instruction reflected by the user input and the position information of the robot model obtained based on the operation detection method can provide a basis for subsequent steps.

[0108] In one possible implementation, see Figure 4 In step S232, obtaining second operation information based on the operation detection method includes:

[0109] S2321, obtaining the robot arm position information; wherein the robot arm position information reflects the relative position information between the robot arm model and the training model.

[0110] It is understandable that the method of obtaining the position information of the robot arm may be to receive data transmitted by the user or to receive position data sent by an input device, but is not limited thereto. Obtaining the position information of the robot arm reflecting the relative position information of the robot arm model and the training model can provide a basis for subsequent steps.

[0111] S2322, determining whether the operating part of the robot arm model in the robot arm position information is located in the first area.

[0112] It can be understood that the method for determining whether the operating part of the robotic arm model in the robotic arm position information is located in the first area can be to determine whether the coordinates of any point on the operating part are within the position range coordinates reflected in the first area, or to determine the distance between any point on the operating part and any point in the first area, etc., but is not limited to this. Determining whether the operating part of the robotic arm model in the robotic arm position information is located in the first area is to determine whether the operating part is likely to change the structure of the training model in the first area when executing the surgical operation instruction. Determining whether the operating part of the robotic arm model in the robotic arm position information is located in the first area can provide a basis for subsequent steps.

[0113] S2323: If the operating part of the robot arm model in the robot arm position information is located in the first area, the robot arm position information and the operation instruction are confirmed as second operation information.

[0114] It can be understood that if the operating part of the robot model in the robot position information is located in the first area, it means that when the robot executes the surgical operation instruction, the structure of the training model in the first area may be changed. Confirming the robot position information and the operation instruction as the second operation information can provide a basis for subsequent steps.

[0115] S233: Confirm the first operation information and the second operation information as operation process information.

[0116] It can be understood that confirming the first operation information and the second operation information as operation process information can provide data support for the user to analyze problems that occur during the operation and provide a basis for subsequent steps.

[0117] S234, receiving an end signal and confirming the training model as a postoperative model; wherein the end signal reflects the completion of the surgical training.

[0118] It can be understood that the end signal is a signal sent after the user ends the training (ends the surgical simulation). After receiving the end signal, confirming the training model as a postoperative model can provide a basis for subsequent steps.

[0119] S235, confirming the operation process information and the postoperative model as postoperative information.

[0120] It can be understood that confirming the operation process information and the postoperative model as postoperative information can provide data support for users to analyze problems that arise during the training process and provide a basis for subsequent steps.

[0121] S300, obtaining a training analysis report based on the postoperative information and the training project information; wherein the training analysis report includes operation process information and surgery result information, and the surgery result information reflects the difference between the three-dimensional digital model reflected by the training target information and the three-dimensional digital model reflected by the postoperative model.

[0122] It can be understood that the training analysis report based on the postoperative information and training project information can be obtained by overlapping the three-dimensional digital model reflected by the training target information and the three-dimensional digital model reflected by the postoperative model according to the positioning marks, and analyzing the area of ​​the overlapping part and the non-overlapping part, or by sending the postoperative information and training project information to the user and receiving the data sent back by the user, etc., but not limited to this. The training analysis report including the operation process information and the operation result information obtained based on the postoperative information and training project information can intuitively display the results of the user's training and the problems encountered by the user during the training process, improve the user's operation proficiency, and improve the problem of doctors' lack of proficiency in operating the surgical arm of the surgical robot.

[0123] In one possible implementation, see Figure 5 , S300, obtaining a training analysis report based on the postoperative information and training project information, including:

[0124] S310: Overlap the first positioning mark in the three-dimensional digital model reflected by the postoperative model with the corresponding second positioning mark in the three-dimensional digital model reflected by the training target information.

[0125] It can be understood that overlapping the first positioning mark in the three-dimensional digital model reflected by the postoperative model with the corresponding second positioning mark in the three-dimensional digital model reflected by the training target information can intuitively analyze the difference between the three-dimensional digital model reflected by the postoperative model obtained after user operation and the three-dimensional digital model reflected by the training target information, and provide a basis for subsequent steps.

[0126] S320: confirm the volume of the non-overlapping portion of the three-dimensional digital model reflected by the postoperative model and the three-dimensional digital model reflected by the training target information as the non-overlapping volume.

[0127] It can be understood that the non-overlapping part of the three-dimensional digital model reflected by the postoperative model and the three-dimensional digital model reflected by the training target information is the difference between the three-dimensional digital model reflected by the postoperative model obtained by the user after the surgical training and the ideal situation. The method of calculating the volume of the non-overlapping part of the three-dimensional digital model reflected by the postoperative model and the three-dimensional digital model reflected by the training target information can be calculated by the volume calculation function provided by the three-dimensional modeling software (such as SolidWorks, AutoCAD, etc.), or by dividing the overlapping part into multiple simple geometric shapes (such as cylindrical slices, cone slices, etc.) by the integral method, and then calculating the volume of each part and adding the results, etc., but not limited to this. Confirming the volume of the non-overlapping part of the three-dimensional digital model reflected by the postoperative model and the three-dimensional digital model reflected by the training target information as the non-overlapping volume can provide a basis for subsequent steps.

[0128] S330: Confirm the volume of the overlapping portion of the three-dimensional digital model reflected by the postoperative model and the three-dimensional digital model reflected by the training target information as the overlapping volume.

[0129] It can be understood that the overlapping part of the three-dimensional digital model reflected by the postoperative model and the three-dimensional digital model reflected by the training target information is the part of the three-dimensional digital model reflected by the postoperative model obtained by the user after the surgical training that meets the ideal situation. The method of calculating the volume of the overlapping part of the three-dimensional digital model reflected by the postoperative model and the three-dimensional digital model reflected by the training target information can be calculated by the volume calculation function provided by the three-dimensional modeling software (such as SolidWorks, AutoCAD, etc.), or by dividing the overlapping part into multiple simple geometric shapes (such as cylindrical slices, cone slices, etc.) by the integral method, and then calculating the volume of each part and adding the results, etc., but not limited to this. Confirming the volume of the overlapping part of the three-dimensional digital model reflected by the postoperative model and the three-dimensional digital model reflected by the training target information as the overlapping volume can provide a basis for subsequent steps.

[0130] S340: Confirm the ratio of the non-overlapping volume to the overlapping volume as surgery result information.

[0131] It can be understood that the closer the ratio of the surgical result information is to 0, the higher the degree of completion of the user's surgery. Confirming the ratio of the non-overlapping volume to the overlapping volume as the surgical result information can intuitively display the degree of completion of the user's surgery and provide a basis for subsequent steps.

[0132] For example, assuming that the non-overlapping volume is 2 square centimeters and the overlapping volume is 50 square centimeters, the surgical result information = 2 / 50 = 0.04.

[0133] S350, confirming the operation result information and the operation process information as a training analysis report.

[0134] It can be understood that confirming the surgical result information and operation process information as a training analysis report can intuitively display the results of user training and the problems encountered by the user during the training process, improve the user's operational proficiency, and improve the doctor's lack of proficiency in operating the surgical arm of the surgical robot.

[0135] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0136] Corresponding to the human-machine collaborative surgery training interaction method described in the above embodiment, the embodiment of the present application also provides a human-machine collaborative surgery training interaction system, and each unit of the system can implement each step of the human-machine collaborative surgery training interaction method. Figure 6 A structural block diagram of a human-machine collaborative surgery training interaction system provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0137] Reference Figure 6 The system comprises:

[0138] An acquisition unit is used to acquire training project information; wherein the training project information includes a training project name, a training model and training target information, the training model is a three-dimensional digital model reflecting the patient's body structure corresponding to the training project name, and the training target information reflects a three-dimensional digital model of the patient's body structure after surgery.

[0139] The first analysis unit is used to obtain postoperative information based on the training project information; wherein the postoperative information includes operation process information and a postoperative model, the operation process information includes information reflecting the operation instructions input by the user, and the postoperative model is a training model after the operation.

[0140] The second analysis unit is used to obtain a training analysis report based on postoperative information and training project information; wherein the training analysis report includes operation process information and surgical result information, and the surgical result information includes the difference between the three-dimensional digital model reflected by the training target information and the three-dimensional digital model reflected by the postoperative model.

[0141] It should be noted that the information interaction, execution process and other contents between the above-mentioned units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0142] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0143] The present application also provides a surgical robot. Figure 7 This is a schematic diagram of the structure of a surgical robot provided in one embodiment of the present application. Figure 7 As shown, the surgical robot of this embodiment includes a control device 6. The control device 6 includes: at least one processor 60 ( Figure 7 Only one is shown), at least one memory 61 ( Figure 7 Only one is shown in the figure) and a computer program 62 stored in the at least one memory 61 and executable on the at least one processor 60. When the processor 60 executes the computer program 62, the surgical robot implements the steps in any of the above-mentioned human-machine collaborative surgery training interaction method embodiments, or implements the functions of the units in the above-mentioned system embodiments.

[0144] Exemplarily, the computer program 62 may be divided into one or more modules / units, which are stored in the memory 61 and executed by the processor 60 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 62 in the control device 6.

[0145] The control device 6 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The control device 6 may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will appreciate that Figure 7 These are merely examples of surgical robots and do not constitute a limitation of the surgical robot. The surgical robot may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the surgical robot may also include input and output devices, network access devices, buses, etc.

[0146] The processor 60 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0147] In some embodiments, the memory 61 may be an internal storage unit of the control device 6, such as a hard disk or memory of the control device 6. In other embodiments, the memory 61 may also be an external storage device of the control device 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the control device 6. Further, the memory 61 may also include both an internal storage unit and an external storage device of the control device 6. The memory 61 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory 61 may also be used to temporarily store data that has been output or is to be output.

[0148] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0149] An embodiment of the present application provides a computer program product. When the computer program product is run on a surgical robot, the surgical robot implements the steps in any of the above-mentioned method embodiments.

[0150] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the surgical robot, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0151] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0152] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0153] In the embodiments provided in the present application, it should be understood that the disclosed human-machine collaborative surgery training interaction system, device and method can be implemented in other ways. For example, the human-machine collaborative surgery training interaction system and device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0154] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0155] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A human-machine collaborative surgery training interactive method, characterized in that: include: Acquire training project information; wherein the training project information includes a training project name, a training model and training target information, the training model is a three-dimensional digital model reflecting the patient's body structure corresponding to the training project name, and the training target information reflects the three-dimensional digital model of the patient's body structure after surgery; Obtaining postoperative information based on the training project information; wherein the postoperative information includes operation process information and a postoperative model, the operation process information includes information reflecting the operation instructions input by the user, and the postoperative model is the training model after the operation; A training analysis report is obtained based on the postoperative information and the training item information; wherein the training analysis report includes the operation process information and the operation result information, and the operation result information reflects the difference between the three-dimensional digital model reflected by the training target information and the three-dimensional digital model reflected by the postoperative model; The three-dimensional digital model reflected by the training model includes at least two first positioning markers and at least one first area, and the three-dimensional digital model reflected by the training target information includes at least two second positioning markers, and the first positioning marker corresponds to one second positioning marker. When all of the first positioning markers coincide with the corresponding second positioning markers, the three-dimensional digital model reflected by the training model can partially coincide with the three-dimensional digital model reflected by the training target information.

2. The human-machine collaborative surgery training interactive method according to claim 1, characterized in that: The obtaining postoperative information based on the training item information includes: Acquire a robotic arm model; wherein the robotic arm model is a three-dimensional digital model reflecting the part of the surgical robot extending into the patient's body, the robotic arm model includes a moving part and an operating part, the moving part reflects the area range of the part of the robotic arm model used to execute the moving instruction, the moving instruction reflects the moving direction and moving speed of the robotic arm model, the operating part reflects the area range of the part of the robotic arm model used to execute the surgical operation instruction, the surgical operation instruction is used to instruct the operating part of the robotic arm model to perform a grasping or cutting action, and the three-dimensional digital model reflected by the robotic arm model does not contact the three-dimensional digital model reflected by the training model in the default position; Obtaining an operation instruction; wherein the operation instruction includes the movement instruction or the surgical operation instruction; The postoperative information is obtained based on the robotic arm model, the operation instructions and the training model.

3. The human-machine collaborative surgery training interactive method according to claim 2, characterized in that: The obtaining the postoperative information based on the robotic arm model, the operation instruction and the training model includes: Determine the instruction type of the operation instruction, and if the operation instruction is a movement instruction, obtain first operation information based on a movement detection method; wherein the first operation information includes the operation instruction reflected by the user input and the position information of the robot arm model; If the operation instruction is a surgical operation instruction, second operation information is obtained based on an operation detection method; wherein the second operation information includes the operation instruction reflected by the user input and the position information of the robot arm model; confirming the first operation information and the second operation information as the operation process information; receiving an end signal, and confirming the training model as the postoperative model; wherein the end signal reflects that the surgical training is completed; The operation process information and the postoperative model are confirmed as the postoperative information.

4. The human-machine collaborative surgery training interactive method according to claim 3, characterized in that: The obtaining of the first operation information based on the mobile detection method includes: Acquire moving direction information; wherein the moving direction information reflects the moving direction of the robotic arm model; Acquire movement speed information; wherein the movement speed information reflects the movement speed of the mechanical arm model; Acquire the position information of the robotic arm; wherein the position information of the robotic arm reflects the relative position information of the robotic arm model and the training model; The first operation information is obtained based on the robot arm position information, the moving speed information and the moving direction information.

5. The human-machine collaborative surgery training interactive method according to claim 4, characterized in that: The obtaining the first operation information based on the robot arm position information, the moving speed information and the moving direction information includes: Determining whether the position of the robotic arm model reflected by the robotic arm position information is within the first area; If the position of the robotic arm model reflected by the robotic arm position information is within the first area, and the moving speed reflected by the moving speed information is greater than a preset moving speed, the robotic arm position information and the moving speed information are confirmed as the first operation information; If the position of the robotic arm model reflected by the robotic arm position information is not within the first area, the speed reflected by the moving speed information is multiplied by m seconds to obtain the moving distance information, and it is determined whether the robotic arm model is within the first area after moving the distance reflected by the moving distance information in the direction reflected by the moving direction information; wherein the moving distance information reflects a distance, m is a positive number, m=0.5; If the robot arm model is located within the first area after moving the distance reflected by the moving distance information in the direction reflected by the moving direction information, the robot arm position information, the moving direction information and the moving speed information are confirmed as the first operation information.

6. The human-machine collaborative surgery training interactive method according to claim 3, characterized in that: The obtaining the second operation information based on the operation detection method includes: Acquire the position information of the robotic arm; wherein the position information of the robotic arm reflects the relative position information of the robotic arm model and the training model; Determining whether the operating part of the robot arm model in the robot arm position information is located in the first area; If the operating part of the robot arm model in the robot arm position information is located in the first area, the robot arm position information and the operation instruction are confirmed as the second operation information.

7. The human-machine collaborative surgery training interactive method according to claim 3, characterized in that: The obtaining of a training analysis report based on the postoperative information and the training item information includes: Overlapping the first positioning mark in the three-dimensional digital model reflected by the postoperative model with the corresponding second positioning mark in the three-dimensional digital model reflected by the training target information; confirming the volume of the non-overlapping portion of the three-dimensional digital model reflected by the postoperative model and the three-dimensional digital model reflected by the training target information as a non-overlapping volume; confirming the volume of the overlapping portion of the three-dimensional digital model reflected by the postoperative model and the three-dimensional digital model reflected by the training target information as the overlapping volume; confirming the ratio of the non-overlapping volume to the overlapping volume as the surgery result information; The operation result information and the operation process information are confirmed as the training analysis report.

8. A human-machine collaborative surgery training interactive system, characterized in that: include: An acquisition unit is used to acquire training project information; wherein the training project information includes a training project name, a training model and training target information, wherein the training model is a three-dimensional digital model reflecting the patient's body structure corresponding to the training project name, and the training target information reflects the three-dimensional digital model of the patient's body structure after surgery; A first analysis unit is configured to obtain postoperative information based on the training item information; wherein the postoperative information includes operation process information and a postoperative model, the operation process information includes information reflecting the operation instructions input by the user, and the postoperative model is the training model after the operation; A second analysis unit is configured to obtain a training analysis report based on the postoperative information and the training item information; wherein the training analysis report includes the operation process information and the operation result information, and the operation result information includes a difference between the three-dimensional digital model reflected by the training target information and the three-dimensional digital model reflected by the postoperative model; The three-dimensional digital model reflected by the training model includes at least two first positioning markers and at least one first area, and the three-dimensional digital model reflected by the training target information includes at least two second positioning markers, and the first positioning marker corresponds to one second positioning marker. When all of the first positioning markers coincide with the corresponding second positioning markers, the three-dimensional digital model reflected by the training model can partially coincide with the three-dimensional digital model reflected by the training target information.

9. A surgical robot comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images