Method, device and controller for determining performance value of robot

Abstract

The present specification provides a robot performance value determination method, device and controller, wherein the method comprises: obtaining an input action amount for driving end activity of a target instrument; predicting, from the perspective of a motion control coordinate system, a first action amount of the input action amount driving the end activity of the target instrument in the case that a performance value of the target instrument reaches a predetermined threshold; determining, from the perspective of a coordinate system of an image obtained by an image acquisition component, a second action amount of the end of the target instrument being active in response to the input action amount; and determining the instrument performance according to the difference between the first action amount and the second action amount. The present scheme can objectively determine the performance value of the target instrument, is not dependent on the experience of the operator, and has high efficiency in performance detection; and can determine the performance value of the target instrument in real time before, during and after operation.

Description

TECHNICAL FIELD

[0001] The present specification relates to the technical field of mechanical equipment, and particularly relates to a performance value determination method and device of a robot and a controller. BACKGROUND

[0002] At present, many operations can be performed by mechanical equipment, and the performance of components on the mechanical equipment determines the accuracy of the operation results to a certain extent. In occasions where the operation accuracy is required to be high, for example, minimally invasive surgery, it is particularly necessary to closely monitor the performance of the mechanical equipment. Therefore, the performance of the mechanical equipment needs to be detected frequently.

[0003] The existing performance detection method is that before the operation, an operator controls the mechanical equipment to "try to operate" (the trial operation may be, for example, to control the mechanical equipment to clamp and cut in the case of being separated from human tissues), and the operator observes the operation results in real time. The operator estimates the performance of the mechanical equipment according to the observed operation results. If the performance of the mechanical equipment meets the standard, the mechanical equipment can be used for formal operation (the formal operation may be, for example, to use the mechanical equipment to clamp and cut in the process of surgery on human tissues), otherwise, the mechanical equipment needs to be replaced and the test needs to be performed again until the mechanical equipment meeting the performance standard is obtained, and then the mechanical equipment meeting the performance standard is used for formal operation. During the formal operation, the operator also needs to determine whether the performance of the mechanical equipment changes according to the real-time observation results. If the performance is reduced to a substandard level, the operator needs to control to stop the operation in time.

[0004] It can be seen that the operation steps of the operator in the existing performance detection method are relatively cumbersome, and the experience of the operator is required to be high. SUMMARY

[0005] The purpose of the embodiments of the present application is to provide a performance value determination method, device and controller of a target instrument to solve the problem that the operation steps of the operator in the existing performance detection method are relatively cumbersome and the experience of the operator is required to be high.

[0006] The first aspect of the present specification provides a performance value determination method of a robot, comprising: acquiring an input action amount for driving the end activity of a target instrument; the target instrument is arranged on a mechanical arm of a target robot; predicting a first action amount of the input action amount driving the end activity of the target instrument in the case that the performance value of the target instrument reaches a predetermined threshold; determining a second action amount of the end of the target instrument responding to the input action amount from an image formed by a signal acquired by an image acquisition component; and determining the instrument performance according to the difference between the first action amount and the second action amount.

[0007] The second aspect of the specification provides a performance value determination device of a robot, comprising: a first acquisition unit configured to acquire an input action amount for driving end activity of a target instrument; the target instrument is arranged on a mechanical arm of a target robot; a first prediction unit configured to predict a first action amount of the input action amount driving end activity of the target instrument in a case where a performance value of the target instrument reaches a predetermined threshold; a first determination unit configured to determine a second action amount of the end of the target instrument moving in response to the input action amount from an image formed by a signal acquired by an image acquisition component; and a second determination unit configured to determine the performance of the instrument according to the difference between the first action amount and the second action amount.

[0008] The third aspect of the specification provides a robot system, comprising: a target robot, the target robot comprising a base, a first mechanical arm and a second mechanical arm, an end of the first mechanical arm being configured to mount a target instrument, an end of the second mechanical arm being configured to mount an image acquisition component; a controller configured to control the first mechanical arm and the second mechanical arm of the target robot to act so that the target instrument performs an operation on an operated object, while acquiring real-time images of the operation process through the image acquisition component; the controller is further configured to perform the performance determination method of the robot of any one of the first aspect.

[0009] The fourth aspect of the specification provides a controller, comprising: a memory and a processor, the processor and the memory being in communication connection with each other, the memory storing computer instructions, and the processor executing the computer instructions to realize the steps of the method of any one of the first aspect.

[0010] The fifth aspect of the specification provides a computer storage medium, the computer storage medium storing computer program instructions, the computer program instructions being executed to realize the steps of the method of any one of the first aspect.

[0011] The method, device and controller for determining the performance value of the robot provided in the specification predict a first action amount in which the end of the target instrument is driven to move when the performance of the target instrument reaches a predetermined threshold according to the input action amount for driving the end of the target instrument, that is, the first action amount is determined from the perspective of the motion control coordinate system; and determine a second action amount in which the end of the target instrument moves in response to the input action amount from the image formed by the signals obtained by the image acquisition component, that is, the second action amount is determined from the perspective of the image coordinate system; and determine the performance value of the target instrument by the difference between the first action amount and the second action amount mapped into the preset coordinate system, which can objectively determine the performance value of the target instrument and does not depend on the experience of the operator. The performance value determination method has low requirements for the operator and high efficiency of performance detection. Moreover, the method can determine the performance value of the target instrument in real time before, during and after operation, so as to timely warn the target instrument with substandard performance and prevent the operation result from being affected by the performance of the instrument, thereby improving the reliability of the operation. BRIEF DESCRIPTION OF DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments described in the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0013] Figure 1 A schematic diagram of a surgical robot system is shown;

[0014] Figure 2 A schematic diagram of a control end device in a surgical robot system is shown;

[0015] Figure 3 A schematic diagram of an image end device in a surgical robot system is shown;

[0016] Figure 4 A schematic diagram of an execution end device in a surgical robot system is shown;

[0017] Figure 5 A flowchart of a performance value determination method of a robot provided in the specification is shown;

[0018] Figure 6 A general structure schematic diagram of a power box is shown;

[0019] Figure 7 A component schematic diagram of a power assembly in the power box is shown;

[0020] Figure 8 A component schematic diagram of a transmission assembly in the power box is shown;

[0021] Figure 9 Fig. 1 shows a schematic diagram of the positional relationship between the target instrument, the image acquisition component, and the object being operated on during the performance of an operation;

[0022] Figure 10 Fig. 2 shows a schematic diagram of the structure inside the tip of the target instrument;

[0023] Figure 11 Fig. 3 shows a schematic diagram of the degrees of freedom of movement of the robotic device;

[0024] Figure 12 Fig. 4 shows a schematic diagram of the coordinate systems associated with the movement control process of the instrument-holding arm of the robot;

[0025] Figure 13 Fig. 5 shows a schematic diagram of the coordinate systems associated with the image acquisition component on the mirror-holding arm of the robot;

[0026] Figure 14 Fig. 6 shows a flowchart of another method for determining the performance value of a robot provided in the present specification;

[0027] Figure 15 Fig. 7 shows a flowchart of yet another method for determining the performance value of a robot provided in the present specification;

[0028] Figure 16 Fig. 8 shows a schematic diagram of the display of instrument performance information;

[0029] Figure 17 Fig. 9 shows a schematic diagram of a performance value determination device for a robot provided in the present specification;

[0030] Figure 18 Fig. 10 shows a schematic diagram of a controller provided in the present specification. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative efforts should fall within the scope of protection of the present application.

[0032] The present specification provides a performance value determination device for an operating device that can automatically determine the performance value of the operating device before, during, and after an operation. In the present specification, “robot” refers to an operating device used to perform an operation.

[0033] For example, an operating device used to perform a minimally invasive surgery can also be referred to as a robot system, such asFigure 1 As shown, the surgical robot consists of a control unit 100, an execution unit 200, and an imaging unit 300. The control unit 100, commonly referred to as the operating table, control console, or doctor's console, is located outside the sterile area of ​​the operating room and is used to send control commands to the execution unit 200. The execution unit 200, also known as the surgical robot device (hereinafter referred to as the surgical robot and the robot device as the robot), is used to control the surgical instruments mounted on its robotic arms to perform specific surgical procedures on the patient according to the control commands. An endoscope may also be mounted on the surgical robot device. The imaging unit 300, commonly referred to as an imaging cart, is used to process the information acquired by the endoscope to form a three-dimensional high-definition image and feed it back to the control unit 100, etc.

[0034] like Figure 2 As shown, the control unit 100, i.e., the doctor's console, is equipped with a main operator hand, an imaging device, and a main controller. The main operator hand detects the surgeon's hand movements, serving as the control signal for the entire surgical robot system. The imaging device provides the surgeon with three-dimensional images of the patient's body detected by the endoscope, providing reliable image information for the surgeon's surgical operations. During surgery, the surgeon sits at the doctor's console and controls the surgical robot and endoscope through the main operator hand. Based on the three-dimensional images of the cavity transmitted back by the imaging device, the surgeon uses hand movements to control the movement of the robotic arm mechanism and surgical instruments on the surgical robot to complete various operations, thereby achieving the goal of performing surgery on the patient. The main controller is the core control element of the surgical robot system, used to control various operations and functions of the surgical robot system.

[0035] like Figure 3 As shown, the image terminal device 300 mainly includes an endoscope (not shown in the figure), an endoscope processor, and a display device. The endoscope includes a tube inserted into the patient's body, an observation lens and an illumination lens at the front end of the tube, an optical fiber, and an eyepiece, used to illuminate the inside of the cavity and acquire a stereoscopic image of the cavity. The endoscope processor is used to process the acquired stereoscopic image of the cavity, and the display device is used to display the processed image in real time.

[0036] like Figure 4 As shown, the execution device 200, i.e., the surgical robot, is located in the sterile area of ​​the operating room. Its main function is to control the surgical instruments mounted on the end of the robotic arm to perform specific surgical procedures on the patient, based on the control commands given by the surgeon, and to carry the endoscope. Within the sterile area, assistant surgeons are usually also present to change the surgical instruments mounted on the surgical robot and assist the surgeon in completing the operation. To ensure patient safety, assistant surgeons typically have higher priority in controlling the surgical robot.

[0037] It should be noted that the "surgery" in the present specification not only includes treatment operations such as resection and suturing of a patient's body by a medical instrument, but also includes operations such as incision, pinching, and puncture for pathological examination of a lesion from the patient's body (i.e., biopsy), that is, the "surgery" in the present specification refers to a means of processing a patient's body for diagnosis and treatment.

[0038] The performance value determination method of the robot provided in the present specification will be mainly described below with reference to the above surgical robot system. As shown in Figure 5 The present specification provides a performance value determination method of a robot, which comprises the following steps:

[0039] S10: Obtain an input action amount for driving the end of the target instrument to move.

[0040] The target instrument is arranged on the mechanical arm of the target robot, and can be detachably arranged or integrally arranged with the robot.

[0041] The "instrument" in the present specification can be understood as a tool in a broad sense. For example, the instrument can be a tool for implementing one of the operations such as shearing and clamping. The "end" of the instrument refers to the end that faces or contacts the target operation object during the operation process, and the "head" of the instrument refers to the end opposite to the "end". For the instrument that is detachably arranged on the robot, the instrument is usually replaced as a whole, and thus the shape and length of the instrument are clear, and the "head" of the instrument is actually the end of the instrument installed on the robot.

[0042] On the one hand, since the operation result is the result of the action of the operation tool on the operation object, during the operation process, the main concern is the interaction process between the end of the operation tool and the operation object, and on the other hand, the field of view of the image acquisition component is limited, and usually only the image acquisition component is adjusted to ensure that the end of the operation tool is within the field of view, and thus the input action amount for driving the end of the instrument to move is obtained in S10.

[0043] The input action amount for driving the end of the instrument to move can be obtained by driving the mechanical arm to move and thus driving the end of the instrument to move, or by keeping the mechanical arm still and only driving the end of the instrument to move, for example, by driving the angle between the two cutting blades of a shearing instrument to be smaller, so as to implement the shearing action.

[0044] For the instrument detachably arranged on the robot, the step S10 of "obtaining the input action amount for driving the end activity of the target instrument" can be obtaining the action amount of any power transmission component on the target robot except the target instrument after the target instrument is mounted on the robot and in the process of the target instrument activity. For example, the power transmission component can be a manipulator for transmitting the operator's body action to the target robot, or an interface arranged on the robot for mounting the target instrument, which can transmit the force on the robot to the end of the target instrument or transmit the electrical control signal on the robot to the target instrument after the target instrument is mounted on the interface, so as to control the end activity of the target instrument.

[0045] The manipulator in the specification can be a manipulator with multiple degrees of freedom as shown in Figure 2 The manipulator in the specification can be a manipulator with multiple degrees of freedom as shown in

[0046] In some embodiments, S10 can include S11 and / or S12.

[0047] S11: obtaining the pose change amount of the manipulator in the robot system, and taking the pose change amount of the manipulator as the input action amount; the manipulator is used to manipulate the end activity of the target instrument.

[0048] The manipulator is a mechanism for converting the body movement of the operator into the control signal of the robot. As shown in Figure 2 The manipulator is a mechanism for converting the body movement of the operator into the control signal of the robot. As shown in

[0049] Inside the robot system, a master-slave consistency control rule is arranged, according to which the pose of the robot will change correspondingly with the pose change of the master manipulator. The master-slave consistency control rule means that the pose change of the master end and the slave end should be consistent, wherein the master end refers to the master manipulator of the operator end, and the slave end refers to the robot device.

[0050] The pose change amount of the manipulator can be the change amount of the manipulator in each degree of freedom. In the process of controlling the pose change of the master manipulator, the first controller will record the real-time pose of the master manipulator and the corresponding time, so that the pose change amount of the manipulator can be obtained from the recording data of the controller.

[0051] S12: Obtain a pose change amount of a power transmission assembly driving the end activity of the target instrument in the robot system, and take the pose change amount of the power transmission assembly as the input action amount; the power transmission assembly is arranged on a target manipulator of the target robot and mechanically connected with the target instrument, and is used for driving the end activity of the target instrument.

[0052] The power transmission assembly is mechanically connected with the target instrument, and the mechanical connection can be direct contact with the connection end of the target instrument or contact with the connection end of the target instrument through other components.

[0053] In the robot system, a plurality of power transmission assemblies, such as gear transmission assemblies, belt transmission assemblies, chain transmission assemblies, etc., can be usually included, and the power transmission assembly in S12 can be any one of the above or a combination of two or more thereof.

[0054] For example, the pose change amount of the gear transmission assembly can be the angle of gear rotation, the number of rotation turns, etc., the pose change amount of the belt transmission assembly can be the distance of any point on the belt relative to a fixed point outside the belt, the number of rotation turns, etc., and the pose change amount of the chain transmission assembly can be the distance of any point on the chain relative to a fixed point outside the chain, the number of rotation turns, etc.

[0055] Since the transmission route of “the input action amount of the manipulator is transmitted to the target instrument” is longer than that of “the input action amount of the power transmission assembly is transmitted to the target instrument”, the overall performance of the target robot or the robot system can be more comprehensively determined according to the input action amount of the manipulator, and the performance of the target instrument can be more accurately determined according to the input action amount of the power transmission assembly.

[0056] In the case where it is necessary to determine the performance of the target robot or the robot system, S10 can include S11.

[0057] In the case where it is necessary to determine the performance of the target instrument installed on the target robot, S10 can include S12. Alternatively, S10 can include S11 and S12, S11 can be performed first to roughly determine the performance of the target instrument according to the input action amount of the manipulator, and in the case where the performance determination result does not reach a predetermined performance value, S12 can not be performed to more accurately determine the performance of the target instrument according to the input action amount of the power transmission assembly.

[0058] In some embodiments, S11 can be obtaining the pose change amount of the manipulator in the process that the operator operates the manipulator to perform a target operation. The target operation can refer to a performance detection operation before formal operation, or an operation in the process of formal operation. The formal operation can be, for example, a surgical operation.

[0059] For example, S11 can include steps S111 and S112.

[0060] S111: When the operator operates the manipulator to move the tip of the target instrument, it is determined whether the change in pose of the manipulator matches a preset action in the set of preset actions.

[0061] The operation of the manipulator can be manual operation or operation by moving other body parts.

[0062] The set of preset actions can include a plurality of preset actions, and the preset actions can be actions that are predetermined and capable of controlling the tip of the target instrument to make a change in pose. For example, the preset actions can be actions that control the shearing instrument to move in a certain direction, actions that control the shears of the shearing instrument to open or close, and the like.

[0063] S111 can be a process in which the operator operates the manipulator to control the tip of the target instrument to move, and the pose of the manipulator is acquired in real time, a determination is made at a predetermined time interval, or a determination is made after each change in pose ends, and it is determined whether any of the changes in pose of the manipulator in the most recent time interval matches a preset action.

[0064] S112: When a preset action is matched, the amount of change in pose of the manipulator corresponding to the preset action is acquired, and the amount of change in pose is used as an input action amount.

[0065] Steps S111 and S112 acquire the pose of the manipulator in real time while the operator operates the manipulator to move the tip of the target instrument, and determine in real time whether the change in pose of the manipulator matches a preset action in the set of preset actions, and when a match is found, the amount of change in pose of the manipulator corresponding to the matched preset action is used as an input action amount, and the performance of the robot or the target instrument mounted on the robot is determined based on the input action amount, so that the performance of the robot or the target instrument can be determined in real time while the operator controls the robot to perform a formal operation, such as a surgeon controlling a surgical robot to perform surgery, so that the performance of the robot or the target instrument suddenly deteriorating to affect the operation result during the formal operation can be prevented.

[0066] In some embodiments, S11 can be acquiring the amount of change in pose of the manipulator during a performance detection operation automatically performed by the robot system before a formal operation is performed. The formal operation can be, for example, a surgical operation.

[0067] For example, S11 can include steps S113, S114, and S115.

[0068] S113: A control instruction sequence that is used to control the tip of the target instrument to make a change in pose is acquired.

[0069] The control system of the robot can pre-store a control instruction sequence set, which can include one or more control instruction sequences, each of which includes control instructions executed in sequence, and the process of executing the control instructions in sequence is the process of controlling the target instrument mounted on the target robot arm to change the pose.

[0070] Since the performance value determination method of the robot provided in the specification needs to obtain the second action amount of the target instrument end in response to the input action amount according to the image acquisition component, the input action amount should be able to drive the target instrument end to change the pose.

[0071] S114: The control instructions in the control instruction sequence are sent to the target robot in sequence, and are fed back to the manipulator of the robot system, so that the pose of the manipulator is synchronized with the action of the target robot.

[0072] Since the master-slave consistency control rule is usually set in the robot system, when the controller controls the robot to change the pose of the target instrument end according to the pre-stored control instruction, the control instruction is also fed back to the manipulator of the robot system, so that the pose of the manipulator is synchronized with the action of the target robot. This is determined by the master-slave consistency control rule of the robot system. The master-slave consistency control rule means that the pose changes of the master and the slave should be consistent, where the master refers to the master operating hand of the operator, and the slave refers to the robot device.

[0073] During the execution of step S114, the appearance of the robot system can be that the pose of the manipulator changes automatically without a person at the operation table, and the pose of the target robot also changes with the pose of the manipulator, so that the pose of the target instrument held by the target robot arm also changes. This series of changes are automatically realized by the robot pose. Of course, it is automatically realized after receiving a control instruction (such as the operator pressing the automatic performance detection button) issued by the operator.

[0074] S115: The pose change amount of the manipulator is used as the input action amount.

[0075] Each control instruction sequence in the control instruction sequence set can be executed in sequence, and each control instruction sequence is executed according to steps S113, S114 and S115. That is, the robot system automatically performs a series of actions, and the performance of the target robot or the target instrument is detected based on the automatically performed series of actions.

[0076] In some embodiments, S12 can be performed before the formal operation, during the formal operation, or after the formal operation.

[0077] For example, S12 can include steps S121 and S122.

[0078] S121: detecting the action amount of the power box during the end activity of the target instrument; wherein the power box is arranged at the end of the target robot arm of the target robot and is detachably connected with the head end of the target instrument.

[0079] S122: taking the action amount of the power box as the input action amount.

[0080] Steps S121 and S122 further limit the power assembly to be the power box on the target robot based on step S12.

[0081] Figure 6 The overall structure of the power box is shown, wherein the power box includes a power assembly 61 and a transmission assembly 62, 63 represents the target instrument.

[0082] The power assembly 61 can include a first connecting piece 611, a plurality of motors 612 and a plurality of first coupling structures 613, in combination Figure 6 and Figure 8 The plurality of motors 612 are arranged on the first connecting piece 611, and the shaft part of each motor 612 is fixedly connected with a first coupling structure 613 through the first connecting piece 611. Each first coupling structure 613 is arranged on the same side of the first connecting piece 611. The shape of the first connecting piece 611 can be any shape, Figure 6 which is only a schematic diagram.

[0083] The transmission assembly 62 includes a second connecting piece 621, a plurality of wire wheels 622 and a plurality of second coupling structures 623, in combination Figure 6 and Figure 7 The shaft part of each wire wheel 622 is fixedly connected with a second coupling structure 623 through the second connecting piece 621. Each second coupling structure 623 is arranged on the same side of the second connecting piece 621.

[0084] The side surface of the second coupling structure 623 away from the second connecting piece 621 is matched with the side surface of the first coupling structure 613 away from the first connecting piece 611, and when the two surfaces are spliced together, the first coupling structure 613 and the second coupling structure 623 can be made to rotate synchronously without an angle difference.

[0085] The power assembly 61 can be fixedly arranged at the end of the mechanical arm of the target robot, and the transmission assembly 62 can be arranged at the head of the instrument. When the target instrument is installed at the end of the mechanical arm of the target robot, each first connecting structure 613 on the power assembly 61 is matched with each second connecting structure 623 on the transmission assembly 62, so that the first connecting structure 613 and the second connecting structure 623 rotate synchronously without an angle difference, thereby transmitting the control force on the mechanical arm to the target instrument.

[0086] Each wire wheel 622 is fixed with an end of a wire, and the other end of the wire passes through the elongated catheter of the target instrument (such as Figure 9 As shown in the figure, M is the target instrument, T is the image acquisition assembly, and X is the object being operated. The target instrument is “connected” with a driven wheel (such as Figure 10 As shown in the figure) at the end of the target instrument. When the mechanical arm of the target robot controls the motor 612 in the power assembly 61 to rotate, it can drive the wire wheel 622 in the transmission assembly 62 to rotate, so as to adjust the length of the wire wound on the wire wheel 622, adjust the tension of the wire segment not wound on the wire wheel 622, and further control the rotation of the driven wheel “connected” with the wire 622, so as to realize the movement of the target instrument in the degree of freedom controlled by the driven wheel, that is, to cause the pose change in the degree of freedom controlled by the driven wheel. For example, Figure 10 The target instrument shown in the figure has the following four degrees of freedom: roll, pitch, yaw, and grip.

[0087] The above-mentioned “connection of the wire and the driven wheel” can be that one end of the wire is connected with a first surface of a driven wheel, and the first surface is a side surface perpendicular to the axis of the driven wheel; or the wire can be wound in a groove of a second surface of the driven wheel, the first end of the wire is fixed on a first wire wheel, the second end of the wire is fixed on a second wire wheel, the movement of the wire is controlled by the rotation of the first wire wheel and the second wire wheel, and the wire has a pre-tightening force, so that the wire can drive the driven wheel to rotate when the wire moves.

[0088] From the above description of the power box, it can be seen that taking the action amount of the power box as the input action amount of the target instrument (that is, as the control amount input to the target instrument) excludes the control error of each part of the robot system except the target instrument, and the control amount input to the target instrument by the robot system can be more accurately measured. Therefore, when the performance value of the target instrument is determined according to the action amount of the power box as the input action amount, the obtained performance value is more accurate.

[0089] On the other hand, in the surgical robot system, the power box is the position where the target instrument is connected with the mechanical arm, and is located outside the body. The volume of the power box can be set to be slightly larger, so that the sensor can be arranged on the power box to obtain the action amount of the power box.

[0090] The motion amount of the power cartridge may be, for example, an output torque of each motor in the power cartridge, an angle of rotation of each motor, a tension on each wire, a torque on each wire reel, and the like.

[0091] S20: predict a first motion amount of the end of the target instrument driven by the input motion amount in a case where a performance value of the target instrument reaches a predetermined threshold.

[0092] In the robot system, the input motion amount has a mapping relationship with the motion of the end of the target instrument. Therefore, step S20 is also to map the input motion amount into a pose change amount of the end of the target instrument in a case where the performance value of the target instrument reaches the predetermined threshold, and to take the mapped pose change amount of the end of the target instrument as the first motion amount.

[0093] In some embodiments, S20 can include the following steps:

[0094] S21: obtain a first pose of the end of the target instrument in a motion coordinate system.

[0095] The motion coordinate system refers to the coordinate system used by the target robot for motion control, and can be referred to the related description of step S41.

[0096] The pose of the end of the target instrument in the motion coordinate system can be the attitude in each degree of freedom direction and the coordinate position. The attitude in each degree of freedom direction may, for example, be the angle in each degree of freedom such as roll, pitch, yaw, grip, and the like as shown in the figure. Figure 10

[0097] The first attitude and the second attitude in the present specification are only used to distinguish different attitudes, and have no other limiting effects.

[0098] S22: predict a second pose of the end of the target instrument in the motion coordinate system after the end of the target instrument is driven by the input motion amount in a case where a performance value of the target instrument reaches a predetermined threshold.

[0099] S23: determine the first motion amount of the end of the target instrument in the motion coordinate system according to the first pose and the second pose.

[0100] S30: determine a second motion amount of the end of the target instrument driven by the input motion amount from an image formed by a signal obtained by the image acquisition component.

[0101] ​In some operations, the real-time operation result cannot be seen by naked eyes. For example, the operation result occurs in a micro area that cannot be recognized by naked eyes, such as an operation result that can be recognized by naked eyes after a certain degree of magnification; or the user is not in the area where the operation result can be seen, such as being in a different room. For such operations, an image acquisition component is arranged at the end of the mechanical arm of the target robot, and the operator understands the real-time operation result through the image formed by the signals collected by the image acquisition component.

[0102] Some image acquisition components can acquire images by themselves, such as image acquisition components with built-in COMS image sensors; some image acquisition components cannot acquire images by themselves and can only acquire intermediate data that can be used to form images, and the images can be obtained by processing the intermediate data by a processor, such as an ultrasonic detector, which can only acquire ultrasonic echo signals and can obtain ultrasonic images by processing the ultrasonic echo signals.

[0103] The image acquisition component can be a lens of an endoscope, an ultrasonic probe of an ultrasonic detector, etc.

[0104] An endoscope is a commonly used medical instrument, which is composed of a cold light source, a light guide structure, and a set of lenses. For example, a laparoscope commonly used in clinical practice is used to transmit images to an ocular lens through a series of optical cylindrical lenses, and imaging is performed through a connected independent camera. According to the imaging principle, an endoscope can be divided into optical mirrors (cylindrical lenses), fiber mirrors, electronic mirrors, etc.

[0105] The change of the pose of the image acquisition component will cause the change of the field of view of the corresponding image. The image corresponding to the pose refers to the image formed by the signals collected when the image acquisition component is in the pose. In a robot system, an automatic pose adjustment method of the image acquisition component is usually arranged, so that the pose of the image acquisition component is automatically adjusted according to the relative position relationship between the object to be operated and the target instrument, and the object to be operated and the target instrument are kept in the field of view of the image acquisition component.

[0106] In combination Figure 1 and Figure 9 In a surgical robot system, the image acquisition component used can be an endoscope. The endoscope is inserted into the cavity of a patient through a small hole on the patient's body, and the pose of the endoscope can be adjusted by a mechanical arm of a surgical robot, or a small motor can be built in the head of the endoscope to adjust the pose of the endoscope. The image terminal device 300 forms a high-magnification stereoscopic image according to the signals collected by the endoscope head and feeds back to the doctor's console.

[0107] In some embodiments, S30 can include:

[0108] S31: acquire an image formed by signals acquired by the image acquisition component in real time, determine a first image before the target instrument tip responds to the input action amount and a second image after the target instrument tip responds to the input action amount.

[0109] The first image before the target instrument tip responds to the input action amount in this step should be understood as an image formed by signals acquired by the image acquisition component at the latest acquisition time before the target instrument tip responds to the input action amount, and the second image after the target instrument tip responds to the input action amount should be understood as an image formed by signals acquired by the image acquisition component at the latest acquisition time after the target instrument tip responds to the input action amount.

[0110] S32: determine a third pose of the target instrument tip in the image coordinate system according to the first image, and determine a fourth pose of the target instrument tip in the image coordinate system according to the second image.

[0111] The image coordinate system refers to a coordinate system established according to the field of view of the image acquisition component, and can be specifically referred to the related description of step S41.

[0112] S33: determine a second action amount of the target instrument tip in the image coordinate system according to the third pose and the fourth pose.

[0113] The pose of the target instrument tip in the image coordinate system can be the attitude in each degree of freedom direction and the coordinate position. The attitude in each degree of freedom direction can be, for example, the angle in each degree of freedom direction such as roll, pitch, yaw, grip, and the like as shown in the following table. Figure 10

[0114] The first attitude and the second attitude in the present specification are only used to distinguish different attitudes, and have no other limiting effects.

[0115] As can be seen, the second action amount in the image coordinate system and the first action amount in the motion coordinate system are in the same dimension, so that the difference between the first action amount and the second action amount can be compared. However, it should be noted that the first action amount and the second action amount are in different coordinate systems and cannot be directly compared. Therefore, step S40 needs to be performed.

[0116] S40: determine the instrument performance according to the difference between the first action amount and the second action amount.

[0117] In some embodiments, the motion coordinate system and the image coordinate system are the same coordinate system, so that S40 can directly calculate the difference between the first action amount and the second action amount, and determine the instrument performance based on the difference.

[0118] ​In some embodiments, the robot device has a complex structure, and the motion coordinate system and the image coordinate system are usually not the same coordinate system. In this case, step S40 can include steps S41, S42 and S43.

[0119] S41: mapping the first motion amount to the preset coordinate system to obtain a third motion amount, and mapping the second motion amount to the preset coordinate system to obtain a fourth motion amount.

[0120] Figure 11 It is shown that Figure 4 The robot device is shown in a schematic view of the degrees of freedom of movement. The robot device can include a base 210 and a robot arm mechanism 220. The robot arm mechanism 220 can include a telescopic arm sub-mechanism 221 and an operating arm sub-mechanism 222. The first end of the telescopic arm sub-mechanism 221 is connected to the base 210, and the telescopic arm sub-mechanism 221 can be elongated or shortened in the radial direction of the base 210. The first end of the operating arm sub-mechanism 222 is connected to the second end of the telescopic arm sub-mechanism 221, and the operating arm sub-mechanism 221 can be bent to switch between an expanded state and a contracted state.

[0121] As shown in Figure 11 The telescopic arm sub-mechanism 221 can include a first cantilever 2211 and a first torsion member 2212. The first torsion member 2212 connects the first end of the first cantilever 2211 to the base 210. The first torsion member 2212 can drive the first cantilever 2211 to rotate in the horizontal plane with the first torsion member 2212 as the center, as shown by the double-headed curve A in Figure 11 This arrangement allows multiple robot arm mechanisms 220 to be contracted together or expanded in the horizontal direction.

[0122] The "rotation in the horizontal plane" described herein can refer to the plane of actual rotational motion having a non-perpendicular angle with the horizontal plane, so that the actual rotational motion has a rotational component in the horizontal plane.

[0123] As shown in Figure 11 The operating arm sub-mechanism 222 can include a second torsion member 2221, a second cantilever 2222, a third torsion member 2223, a third cantilever 2224, a fourth torsion member 2225, a fourth cantilever 2226, a fifth torsion member 2227, and a fifth cantilever 2228. The instrument M is mounted on the fifth cantilever 2228.

[0124] The second cantilever 2222 is located below the first cantilever 2211, and the second torsion member 2221 connects the second end of the first cantilever 2211 and the first end of the second cantilever 2222. The second torsion member 2221 can drive the second end of the second cantilever 2222 to move towards or away from the base 210 in the vertical plane, as shown by the double-headed curve B in Figure 11 .

[0125] The third cantilever 2224 is located on the side of the second end of the second cantilever 2222 away from the base 210, and intersects with the second cantilever 2222 at a fixed included angle (for example, the fixed included angle in Figure 11 is an acute angle). The third torsion member 2223 connects the first end of the third cantilever 2224 and the second end of the second cantilever 2222, and the third torsion member 2223 can drive the third cantilever 2224 to rotate around its own axis, as shown by the double-headed curve C in Figure 11 .

[0126] The fourth torsion member 2225 connects the second end of the third cantilever 2224 and the first end of the fourth cantilever 2226. The fourth torsion member 2225 can drive the fourth cantilever 2226 to move, so as to change the included angle between the third cantilever 2224 and the fourth cantilever 2226, as shown by the double-headed curve D in Figure 11 .

[0127] The fifth torsion member 2227 connects the second end of the fourth cantilever 2226 and the first end of the fifth cantilever 2228, and the second end of the fifth torsion member 2227 is provided with a mechanical claw to clamp the target instrument to perform clamping, cutting, shearing and other surgical operations. The fifth torsion member 2227 drives the fifth cantilever 2228 to move, so as to change the included angle between the fourth cantilever 2226 and the fifth cantilever 2228.

[0128] In the above-mentioned second torsion member 2221, third torsion member 2223, fourth torsion member 2225 and fifth torsion member 2227, a torsion motor can be arranged respectively, and these torsion motors are electrically connected with the controller, so that the controller can control the torsion members to drive the cantilevers to move, and further drive the pose of the instrument M to change. The above-mentioned torsion members are the joints of the mechanical arm.

[0129] Figure 11 Only a structural schematic diagram of a robot device is given, and in some embodiments, the degrees of freedom of the robot device and the number of mechanical arms can be more or less than Figure 12 the robot device.

[0130] Figure 4 It is shown that Figure 11 or Figure 12A schematic diagram of various coordinate systems associated with the motion control process of the holding arm (i.e. the mechanical arm mounting the target instrument) of the robot shown. The following takes the robot device as an example to illustrate the various coordinate systems associated with the motion control process of the holding arm.

[0131] As shown in Figure 11 , x0y0z0 is a first base coordinate system of the robot device as a whole, which is a coordinate system established with a point on the base as the coordinate origin; x1y1z1 is a second base coordinate system of the target mechanical arm holding the target instrument, which is a coordinate system established with a point on the telescopic arm sub-mechanism 221, for example, a coordinate system established with a point on the telescopic arm 221 away from the base; x m y m z m is a mechanical arm end coordinate system, which is a coordinate system established with the end of the mechanical arm (the end refers to the end away from the base on the control signal conduction path), for example, a coordinate system established with a point on the end of the fifth cantilever in Figure 13 ; x q y q z q is an instrument end coordinate system, which is a coordinate system established with a point on the end of the instrument (the end refers to the end away from the mechanical arm).

[0132] In the motion control process of the holding arm, the sensors arranged in the joints of the holding arm collect the position information of the joints, and the pose and coordinate position of the end of the target instrument in the instrument end coordinate system are determined according to the collection results. The pose and coordinate pose can be mapped to various coordinate systems in the order of “instrument end coordinate system -> mechanical arm end coordinate system -> second base coordinate system -> first base coordinate system” in step S41.

[0133] Figure 4 A schematic diagram of various coordinate systems associated with the image acquisition component on the holding arm (i.e. the mechanical arm mounting the image acquisition component) of the robot shown. The following takes the robot device as an example to illustrate the various coordinate systems associated with the image acquisition component. Figure 11 or Figure 13 A schematic diagram of various coordinate systems associated with the image acquisition component on the holding arm (i.e. the mechanical arm mounting the image acquisition component) of the robot shown. The following takes the robot device as an example to illustrate the various coordinate systems associated with the image acquisition component.

[0134] As shown in Figure 12 , x0y0z0, x1y1z1, x m y m z m The coordinate systems shown are the same as the coordinate systems with the same names in Figure 5 , x T y T z Tis the image acquisition component coordinate system, which can be a coordinate system established with a point on the visual axis of the image acquisition component, on the side of the mechanical arm, as the coordinate origin; x C y C z C is the image acquisition component front end coordinate system, which can be a coordinate system established with the front end of the image acquisition component (for example, the front end of the lens of an endoscope, the ultrasonic signal emission point of an ultrasonic signal transceiver) as the coordinate origin; x a y a z a is the field of view projection surface coordinate system, which can be a coordinate system established on the projection surface of the target instrument end in each projection surface (the projection surface refers to the cross section of the field of view perpendicular to the visual axis) of the field of view of the image acquisition component, and the coordinate origin can be a point on the projection screen of the visual axis.

[0135] In the motion control process of the mirror holding arm, the sensors arranged in each joint of the mirror holding arm collect the position information of the joint, and the coordinate position of the coordinate origin of the image acquisition component coordinate system is determined according to the collection result. The coordinate position can be gradually mapped to each coordinate system in the order of “image acquisition component coordinate system -> mechanical arm end coordinate -> second base coordinate system -> first base coordinate system” by step S41.

[0136] The image acquisition component can acquire the real-time image of the target instrument processing the operated object through binocular vision technology or infrared vision technology, and the image can contain depth information. Based on the rich information collected by the image, the pose and coordinate position of the end of the target instrument in the field of view projection surface coordinate system can be determined, and then the “pose and coordinate position in the field of view projection surface coordinate system” is mapped to the “pose and coordinate position in the image acquisition component front end coordinate system” according to the depth information of the image. Then, the “pose and coordinate position in the image acquisition component front end coordinate system” is mapped to the “pose and coordinate position in the image acquisition component coordinate system” according to the lens structure parameters of the image acquisition component, and then the mapping to other coordinate systems can be performed according to the mapping mode of the last paragraph.

[0137] As can be seen from the above description, the coordinate system in S41 can be any one of the above first base coordinate system, second base coordinate system, mechanical arm end coordinate system, and instrument end coordinate system.

[0138] It should be noted that, Figure 5The method shown is to first determine the first action amount and the second action amount, then map the first action amount and the second action amount to a predetermined coordinate system to obtain a third action amount and a fourth action amount respectively, and then perform a difference operation on the mapping results of the third action amount and the fourth action amount. In some embodiments, the pose information and the position coordinates used to determine the first action amount can also be mapped to a predetermined coordinate system first, and then the third action amount is obtained according to the mapping result. Similarly, the pose information and the position coordinates used to determine the second action amount can also be mapped to a predetermined coordinate system first, and then the fourth action amount is obtained according to the mapping result. Finally, the third action amount and the fourth action amount are subjected to a difference operation. This implementation is Figure 14 The equivalent implementation of the method shown should be determined to belong to the protection scope of the present application.

[0139] S42: Calculate the difference between the third action amount and the fourth action amount.

[0140] According to the manner of obtaining the action amount in the present specification, the difference calculated in S42 can be an angle difference or a coordinate difference.

[0141] S43: Determine the performance value of the target instrument according to the difference.

[0142] Specifically, S43 can set a respective numerical range for each dimension in the action amount, and each numerical range corresponds to a performance value. Thus, for a dimension, the performance value of the target instrument can be determined according to the numerical range in which the difference value is located.

[0143] In some cases, different performance values can be obtained according to the differences of different dimensions, then the performance value of the target instrument can be taken as the worst result, or the performance values corresponding to each dimension can be weighted and summed to obtain the performance value of the target instrument. Of course, other ways can also be used, which will not be enumerated one by one in the present specification.

[0144] The method for determining the performance value of the robot provided in the specification predicts a first action amount in which the end of the target instrument is driven to move when the performance of the target instrument reaches a predetermined threshold according to an input action amount for driving the end of the target instrument to move, that is, the first action amount is determined from the perspective of the motion control coordinate system; and determines a second action amount in which the end of the target instrument moves in response to the input action amount from an image formed from a signal obtained by the image acquisition component, that is, the second action amount is determined from the perspective of the image coordinate system; and determines the performance value of the target instrument by the difference between the first action amount and the second action amount mapped into a preset coordinate system, which can objectively determine the performance value of the target instrument without relying on the experience of the operator, and the performance detection efficiency is relatively high. Moreover, the method can determine the performance value of the target instrument in real time before, during and after operation, so as to timely warn the target instrument with substandard performance, prevent the operation result from being affected by the performance of the instrument, and improve the reliability of the operation.

[0145] In some embodiments, as shown in S40, after S40, the following steps S50 to S70 can also be included. Figure 15

[0146] S50: predicting a continuation value of the service life of the target instrument from the current time according to the performance value of the target instrument.

[0147] The "continuation value of the service life" refers to the life value increased on the basis of the current rated life.

[0148] The service life of the target instrument can refer to the remaining use times or the remaining use time of the current target instrument.

[0149] S60: obtaining the latest service life of the target instrument from the storage device built in the target instrument.

[0150] The storage device built in the target instrument can be a storage physical device arranged in the target instrument, or a label (such as a two-dimensional code mark) pointing to the network memory arranged on the surface of the target instrument.

[0151] Step S60 can obtain the latest service life from the storage device built in the target instrument through any existing short-range wireless communication technology.

[0152] S70: adjusting the service life of the target instrument according to the continuation value of the service life, and writing the adjusted service life into the storage device built in the target instrument as the latest service life.

[0153] ​The target instrument and the robot device are usually separately arranged, and there are many types of target instruments, and each target instrument can perform different operations. The type of the target instrument can be selected according to the actual operation needs, and the selected target instrument can be installed on the robot device. After the target instrument is produced, the manufacturer can determine the rated service life of the target instrument according to the test results before leaving the factory, and the rated service life can be stored in the storage device built in the target instrument.

[0154] Before each use of the target instrument, the latest service life of the target instrument can be obtained from the storage device built in the target instrument. Whether the current use exceeds the latest service life of the target instrument is determined. If not, the target instrument is used. If it exceeds, it is not used. Alternatively, if it exceeds the latest service life obtained from the built-in storage device, the latest service life of the target instrument is further adjusted by the method corresponding to steps S50 to S70, and then it is determined again whether the current use exceeds the adjusted latest service life. If not, the target instrument is continued to be used.

[0155] By adjusting the service life of the target instrument, the instrument with good performance when reaching the rated service life can be fully utilized, and the instrument with poor performance before reaching the rated service life can be marked to prevent the instrument with poor performance from affecting the operation result.

[0156] In some embodiments, as shown in FIG. 1, before S10, the following steps S100 to S130 can also be included. Figure 5

[0157] S100: Obtain the first type to which the target instrument belongs from the storage device built in the target instrument.

[0158] The storage device built in the target instrument can be a storage entity arranged in the target instrument, or a label (such as a two-dimensional code mark) pointing to the network memory arranged on the surface of the target instrument. That is, the first type to which the target instrument belongs and the latest service life of the target instrument can be stored in the same storage device. Of course, the storage device can also store other information of the target instrument.

[0159] Step S100 can obtain the first type to which the target instrument belongs from the storage device built in the target instrument by any existing short-distance wireless communication technology.

[0160] S110: Determine the second type to which the target instrument belongs from the image formed by the signal obtained by the image acquisition component.

[0161] S120: Determine whether the first type and the second type match.

[0162] ​S130: In the case of matching, the input motion amount for driving the end activity of the target instrument is executed.

[0163] By judging whether the category stored in the target instrument built-in memory matches the category determined according to the image, only in the case of matching, the performance detection method is executed. Figure 16 The performance detection method shown can avoid the case of mistaken information of the target instrument.

[0164] After determining the performance value of the target instrument in step S40, the performance value can be presented to the operator in the form of visual feedback. For example, the instrument performance information shown in Figure 5 may be displayed on the display of the operating table, and the instrument performance information can include the identification of the instrument (such as instrument 1, instrument 2), performance parameters, fault types, etc.; or the instrument performance information can be displayed on the display of the image terminal device 300. In some embodiments, a sound prompting device can also be provided on the operating table and / or the image terminal device 300 to present the fault information in the form of sound; or a light prompting device can be provided to present the fault information in the form of light.

[0165] The fault type can be divided into performance fault and structure fault, wherein the performance fault can mean that the performance value reaches a predetermined performance value threshold. The structure fault can be determined by the following method: obtaining an image formed by signals collected by the image acquisition component, determining the category of the target instrument in the image; identifying the features of the end of the target instrument in the image according to the category of the target instrument, which can include color, contour, surface texture, etc.; matching the features of the end of the target instrument with the fault feature template of the instrument of the category; when the fault feature template is matched, the fault type of the target instrument is determined as a structure fault. The fault type corresponding to the fault feature template can also be taken as the fault type of the target instrument.

[0166] The fault feature template of the instrument can be the target of one or more of the following faults: wire off-wheel, wire end fluff, instrument material cracking, electric knife instrument material burning, instrument insulation sleeve separation.

[0167] The present specification provides a performance value determination device of a robot, which can be used to implement Figure 17 the performance value determination method of the robot shown. As Figure 5 shown, the device includes a first acquisition unit 10, a first prediction unit 20, a first determination unit 30, and a second determination unit 40.

[0168] The first acquisition unit 10 is configured to acquire an input motion amount for driving an end of a target instrument; the target instrument is arranged on a mechanical arm of a target robot. The first prediction unit 20 is configured to predict a first motion amount of the input motion amount for driving the end of the target instrument in a case where a performance value of the target instrument reaches a predetermined threshold. The first determination unit 30 is configured to determine a second motion amount of the end of the target instrument in response to the input motion amount from an image formed by a signal acquired by an image acquisition component. The second determination unit 40 is configured to determine the performance of the instrument according to a difference between the first motion amount and the second motion amount.

[0169] In some embodiments, the second determination unit 40 comprises: a mapping subunit configured to map the first motion amount to a preset coordinate system to obtain a third motion amount, and map the second motion amount to the preset coordinate system to obtain a fourth motion amount; a calculation subunit configured to calculate a difference between the third motion amount and the fourth motion amount; and a first determination subunit configured to determine the performance value of the target instrument according to the difference.

[0170] In some embodiments, the first prediction unit comprises: a first acquisition subunit configured to acquire a first pose of the end of the target instrument in a motion coordinate system; the motion coordinate system is a coordinate system used for motion control of the target robot; a prediction subunit configured to predict a second pose of the end of the target instrument in the motion coordinate system after the end of the target instrument is driven to move by the input motion amount in a case where the performance value of the target instrument reaches a predetermined threshold; and a second determination subunit configured to determine the first motion amount of the end of the target instrument in the motion coordinate system according to the first pose and the second pose.

[0171] In some embodiments, the first determination unit comprises: a second acquisition subunit configured to acquire, in real time, an image formed by a signal acquired by the image acquisition component, determine a first image before the end of the target instrument is driven to move by the input motion amount, and a second image after the end of the target instrument is driven to move by the input motion amount; a third determination subunit configured to determine a third pose of the end of the target instrument in an image coordinate system according to the first image, and determine a fourth pose of the end of the target instrument in the image coordinate system according to the second image; and a fourth determination subunit configured to determine the second motion amount of the end of the target instrument in the image coordinate system according to the third pose and the fourth pose.

[0172] In some embodiments, the first obtaining unit comprises: a third obtaining subunit, configured to obtain a pose change amount of a manipulator in the robot system, and take the pose change amount of the manipulator as the input action amount; the manipulator is used to operate the end activity of the target instrument; or a fourth obtaining subunit, configured to obtain a pose change amount of a power transmission assembly in the robot system that drives the end activity of the target instrument, and take the pose change amount of the power transmission assembly as the input action amount; the power transmission assembly is arranged on a target mechanical arm of the target robot, and is mechanically connected with the target instrument, and is used to drive the end activity of the target instrument.

[0173] In some embodiments, the third obtaining subunit comprises: an identifying subunit, configured to identify whether the pose change of the manipulator matches a preset action in a preset action set when the operator operates the manipulator to make the end activity of the target instrument; and a fifth obtaining subunit, configured to obtain a pose change amount of the manipulator corresponding to the preset action in the case of matching the preset action, and take the pose change amount as the input action amount.

[0174] In some embodiments, the first obtaining unit comprises: a sixth obtaining subunit, configured to obtain a pre-stored control instruction sequence, the control instruction sequence being used to control the end of the target instrument to generate a pose change; a sending subunit, configured to send control instructions in the control instruction sequence to the target robot in sequence, and feed back to the manipulator of the robot system at the same time, so that the pose of the manipulator is synchronized with the action of the target robot; and a fifth determining subunit, configured to take the pose change amount of the manipulator as the input action amount.

[0175] In some embodiments, the first obtaining unit comprises: a detecting subunit, configured to detect the action amount of the power box during the end activity of the target instrument; and a sixth determining subunit, configured to take the action amount of the power box as the input action amount; the power box is arranged at the end of the target mechanical arm of the target robot, and is detachably connected with the head end of the target instrument.

[0176] In some embodiments, the device further comprises: a second obtaining unit, configured to obtain a first category to which the target instrument belongs from a storage device built in the target instrument; a third determining unit, configured to determine a second category to which the target instrument belongs from an image formed by the signal obtained by the image acquisition component; a fourth determining unit, configured to determine whether the first category and the second category match; and in the case of matching, the first obtaining unit performs the obtaining of the input action amount for driving the end activity of the target instrument.

[0177] In some embodiments, the device further comprises a second prediction unit configured to predict a continuation value of the service life of the target instrument from the current time according to the performance value of the target instrument; a third acquisition unit configured to acquire the latest service life of the target instrument from a storage device built in the target instrument; and an adjustment unit configured to adjust the service life of the target instrument according to the continuation value of the service life, and write the adjusted service life into the storage device built in the target instrument as the latest service life.

[0178] In some embodiments, the device further comprises a fourth acquisition unit configured to acquire an image formed according to a signal collected by the image acquisition component; a fifth determination unit configured to determine the type of the target instrument in the image; identify a feature of the end of the target instrument in the image according to the type of the target instrument; a matching unit configured to match the feature of the end of the target instrument with a failure feature template of the instrument of the type; and a sixth determination unit configured to determine the failure type of the target instrument as the structural failure when the failure feature template is matched.

[0179] In some embodiments, the device further comprises a judgment unit configured to judge whether the performance value of the target instrument reaches a preset performance value threshold; a seventh determination unit configured to determine that the target instrument has a performance failure when the performance value reaches the preset performance value threshold; and a presentation unit configured to present the failure type of the target instrument to an operator, the failure type including the performance failure and / or the structural failure.

[0180] The specific details of the performance value determination device of the robot can be found in Figure 1 The related descriptions and effects in the corresponding embodiments are understood, and will not be repeated here.

[0181] The present specification provides a robot system, including a target robot and a controller. The controller can be arranged in the target robot, or arranged in an operation table, or arranged independently of the target robot and the operation table.

[0182] The target robot includes a base, a first mechanical arm, and a second mechanical arm, an end of the first mechanical arm is configured to mount a target instrument, an end of the second mechanical arm is configured to mount an image acquisition component; a controller configured to control the first mechanical arm and the second mechanical arm of the target robot to act, so that the target instrument performs an operation on an operated object, and simultaneously acquires a real-time image of the operation process through the image acquisition component; the controller is further configured to execute the performance determination method of the robot.

[0183] The target robot can be Figure 18 the robot device 200 in The description of the robot device 200 can refer to the description in other parts of the present specification, and will not be repeated here.

[0184] The present specification also provides a controller, such asFigure 18 As shown, the controller may include a processor 1801 and a memory 1802, wherein the processor 1801 and the memory 1802 can be connected via a bus or other means. Figure 17 Taking the example of a connection between China and Israel via a bus.

[0185] Processor 1801 can be a Central Processing Unit (CPU). Processor 1801 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.

[0186] Memory 1802, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the robot performance value determination method in this embodiment of the invention (e.g., Figure 5 The processor 1801 comprises a first acquisition unit 10, a first prediction unit 20, a first determination unit 30, a mapping unit 40, a calculation unit 50, and a second determination unit 60. The processor 1801 executes various functional applications and data classification by running non-transitory software programs, instructions, and modules stored in the memory 1802, thereby implementing the robot performance value determination method in the above method embodiments.

[0187] The memory 1802 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor 1801, etc. Furthermore, the memory 1802 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 1802 may optionally include memory remotely located relative to the processor 1801, and these remote memories may be connected to the processor 1801 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0188] The one or more modules are stored in the memory 1802, and when executed by the processor 1801, they perform the following: Figure 5 The method for determining the performance values ​​of the robot in the illustrated embodiment.

[0189] The specific details of the above controller can be found in Figure 5 The relevant description and effects in the corresponding embodiments are understood, and will not be repeated here.

[0190] The present specification also provides a computer storage medium, which stores computer program instructions, and the computer program instructions are executed to implement ​ The steps of the corresponding embodiments.

[0191] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The program can be stored in a computer readable storage medium, and when the program is executed, the processes of the above-mentioned embodiments of the method can be included. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc. The storage medium can also include a combination of the above-mentioned types of memories.

[0192] Each embodiment in the present specification is described in a progressive manner, and the same or similar parts between each embodiment can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, for the hardware+program type embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0193] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims can be performed in an order different than the order in which the acts or steps are recited in the embodiments and still achieve desirable results. In addition, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous.

[0194] Those skilled in the art will also know, in addition to implementing the controller in the form of a pure computer readable program code, it is also possible to implement the same function by logically programming the method steps in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers, etc. Therefore, such a controller can be considered as a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, even the means for implementing various functions can be considered as both a software module implementing a method and a structure within a hardware component.

[0195] The above only describes examples of the embodiments of the present specification, and is not intended to limit the embodiments of the present specification. The embodiments of the present specification can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of the present specification shall be included in the scope of claims of the embodiments of the present specification.

Claims

1. A method for determining the performance values ​​of a robot, characterized in that, include: The pose change of the manipulator on the control panel used to drive the end effector of the target device is acquired as the input motion quantity; the target device is mounted on the robotic arm of the target robot. The first action amount that drives the end effector of the target device when the performance value of the target device reaches a predetermined threshold is predicted. From the image formed by the signal acquired by the image acquisition component, a second action amount is determined in which the end of the target instrument moves in response to the input action amount; The performance of the device is roughly determined based on the difference between the first and second stroke amounts. When the roughly determined performance of the instrument reaches the predetermined performance value, the pose change of the power transmission component used to drive the end effector of the target instrument is obtained as the input motion quantity; the power transmission component is set on the target robotic arm of the target robot and is mechanically connected to the target instrument to drive the end effector of the target instrument. The first action amount that drives the end effector of the target device when the performance value of the target device reaches a predetermined threshold is predicted. From the image formed by the signal acquired by the image acquisition component, a second action amount is determined in which the end of the target instrument moves in response to the input action amount; The performance of the device can be more accurately determined by the difference between the first and second motion quantities.

2. The method according to claim 1, characterized in that, The device performance is determined based on the difference between the first and second stroke amounts, including: The first motion quantity is mapped to the preset coordinate system to obtain the third motion quantity, and the second motion quantity is mapped to the preset coordinate system to obtain the fourth motion quantity; Calculate the difference between the third and fourth motion quantities; Based on the difference, the performance value of the target device is determined.

3. The method according to claim 1, characterized in that, Predicting that, when the performance value of the target device reaches a predetermined threshold, the first action amount that drives the end effector movement of the target device, as indicated by the input action amount, includes: Obtain the first pose of the end effector of the target device in the motion coordinate system; the motion coordinate system is the coordinate system used by the target robot when performing motion control; Predicting the second pose of the end effector in the motion coordinate system after the input motion quantity drives the end effector of the target device to move when the performance value of the target device reaches a predetermined threshold. The first motion of the end effector of the target instrument in the motion coordinate system is determined based on the first and second poses.

4. The method according to claim 1, characterized in that, From the image formed by the signals acquired by the image acquisition component, a second action quantity is determined in response to the input action quantity, which is the movement of the end effector of the target instrument. The image formed by the signal acquired by the real-time image acquisition component is used to determine a first image before the target instrument end responds to the input action amount and a second image after the target instrument end responds to the input action amount; The third pose of the target instrument end in the image coordinate system is determined based on the first image, and the fourth pose of the target instrument end in the image coordinate system is determined based on the second image. The second motion of the end effector of the target instrument in the image coordinate system is determined based on the third and fourth poses.

5. The method according to claim 1, characterized in that, The pose change of the manipulator on the control panel used to drive the end effector movement of the target instrument is acquired as the input motion quantity, including: When the operator manipulates the manipulator to move the end effector of the target instrument, it identifies whether the positional change of the manipulator matches the preset action in the preset action set; When a preset action is matched, the pose change of the manipulator corresponding to the preset action is obtained, and the pose change is used as the input action amount.

6. The method according to claim 1, characterized in that, The pose change of the manipulator on the control panel used to drive the end effector movement of the target instrument is acquired as the input motion quantity, including: Obtain a pre-stored sequence of control commands, which is used to control the end effector of the target instrument to produce a pose change; The control commands in the control command sequence are sent sequentially to the target robot and simultaneously fed back to the manipulator of the robot system so that the manipulator's pose moves in sync with the target robot. The change in the position of the manipulator is used as the input motion quantity.

7. The method according to claim 1, characterized in that, The pose change of the power transmission component used to drive the end effector of the target instrument is acquired as the input motion quantity, including: During the end-effector movement of the target device, the amount of motion of the power box is detected; wherein, the power box is located at the end of the target robotic arm of the target robot and is detachably connected to the head end of the target device. Use the motion quantity of the power box as the input motion quantity.

8. The method according to claim 1, characterized in that, Before acquiring the input motion quantity used to drive the end effector's movement, the following is included: Obtain the first category to which the target device belongs from the storage device built into the target device; From the image formed by the signals acquired by the image acquisition component, determine the second category to which the target instrument belongs; Determine whether the first category and the second category match; In the case of a match, the input motion quantity for driving the end effector of the target instrument is obtained.

9. The method according to claim 2, characterized in that, After determining the performance value of the target device based on the difference, the process further includes: Predict the extension of the target instrument's service life from the current moment based on the target instrument's performance values; Obtain the latest lifespan of the target device from its built-in storage devices; The lifespan of the target device is adjusted based on the extended lifespan value, and the adjusted lifespan is written into the target device's built-in storage device as the latest lifespan.

10. The method according to claim 1, characterized in that, Also includes: Acquire an image formed from signals collected by the image acquisition component; Determine the type of target instrument in the image; Identify the features of the end of the target instrument in the image based on the type of target instrument; Match the features of the target device's end with the fault feature template of the device of that type; When a fault feature template is matched, the fault type of the target instrument is determined to be a structural fault.

11. The method according to claim 10, characterized in that, Also includes: Determine whether the performance value of the target device reaches a preset performance value threshold; If the condition is met, it is determined that the target instrument has a performance failure; Present the operator with the types of faults of the target instrument, including performance faults and / or structural faults.

12. A device for determining the performance value of a robot, characterized in that, include: The first acquisition unit is used to acquire the pose change of the manipulator on the operating table used to drive the end effector of the target device as the input motion quantity; the target device is set on the robotic arm of the target robot. The first prediction unit is used to predict the first action amount that drives the end effector of the target device when the performance value of the target device reaches a predetermined threshold. The first determining unit is used to determine, from the image formed by the signal acquired by the image acquisition component, a second action amount that the end of the target instrument moves in response to the input action amount; The second determining unit is used to roughly determine the performance of the device based on the difference between the first action amount and the second action amount. The device is also used for: When the roughly determined performance of the instrument reaches the predetermined performance value, the pose change of the power transmission component used to drive the end effector of the target instrument is obtained as the input motion quantity; the power transmission component is set on the target robotic arm of the target robot and is mechanically connected to the target instrument to drive the end effector of the target instrument. The first action amount that drives the end effector of the target device when the performance value of the target device reaches a predetermined threshold is predicted. From the image formed by the signal acquired by the image acquisition component, a second action amount is determined in which the end of the target instrument moves in response to the input action amount; The performance of the device can be more accurately determined by the difference between the first and second motion quantities.

13. A robot system, characterized in that, include: The target robot includes a base, a first robotic arm, and a second robotic arm. The end of the first robotic arm is used to mount a target instrument, and the end of the second robotic arm is used to mount an image acquisition component. The controller is used to control the movements of the first and second robotic arms of the target robot so that the target robot can perform operations on the manipulated object, and at the same time, it can acquire real-time images of the operation process through the image acquisition component. The controller is also used to perform the method for determining the performance value of the robot as described in any one of claims 1 to 11.

14. The system according to claim 13, characterized in that, Also includes: The manipulator is used by the operator to control the movement of the first and second robotic arms of the target robot, so as to control the target instrument to perform operations on the manipulated object, and at the same time, the image acquisition component acquires real-time images of the operation process.

15. The system according to claim 13, characterized in that, It also includes a power box, which comprises: A power assembly is fixedly mounted at the end of the first robotic arm; the power assembly includes a first connector, multiple motors, and multiple first connecting structures. The multiple motors are mounted on the first connector, and the shaft of each motor passes through the first connector and is fixedly connected to a first connecting structure; each first connecting structure is located on the same side of the first connector. A transmission assembly is fixedly installed at the head section of the target instrument; the transmission assembly includes a second connector, multiple lead wheels, and multiple second connecting structures, with the shaft of each lead wheel passing through the second connector and fixedly connected to a second connecting structure; each second connecting structure is located on the same side of the second connector; When the target device is installed at the end of the first robotic arm, when the side surface of the second connecting structure on the power assembly away from the second connector is spliced ​​with the side surface of the first connecting structure on the transmission assembly away from the first connector, the first connecting structure and the second connecting structure rotate synchronously, thereby driving the lead wheel and the motor to rotate synchronously, thus transmitting the control force on the first robotic arm to the target device. Each thread wheel has one end of a thread fixed on it. The other end of the thread passes through the tube on the target instrument and connects to the driven wheel at the end of the target instrument. When the motor in the power assembly is selected, it drives the thread wheel in the transmission assembly to rotate, thereby adjusting the tension on the thread and / or controlling the movement of the thread. In turn, the thread drives the driven wheel to rotate, so that the target instrument can move in the degree of freedom controlled by the driven wheel.

16. A controller, characterized in that, include: A memory and a processor, the processor and the memory being communicatively connected to each other, the memory storing computer instructions, the processor executing the computer instructions to implement the steps of the method according to any one of claims 1 to 11.

17. A computer storage medium, characterized in that, The computer storage medium stores computer program instructions, which, when executed, implement the steps of the method according to any one of claims 1 to 11.

Images