Robotic system and method of controlling the same, medium

Abstract

A robot system and a control method thereof, a medium, the robot system comprising: an output device configured to output indication information comprising at least one operation action indication, the at least one operation action indication corresponding to at least one operation action of an input device one-to-one, the at least one operation action corresponding to at least one function of the robot system; the input device being configurable in a follow mode or a controlled mode, in the case of being configured in the follow mode, being in a coupled state with a manipulator assembly to make the manipulator assembly follow the input device movement, in the case of being configured in the controlled mode, being in a decoupled state with the manipulator assembly and being executable to perform an operation action indicated by any one operation action indication; a control device configured to, in the case of the input device being configured in the controlled mode, detect a current operation action of the input device, determine a function corresponding to the current operation action, and control the robot system to activate the function corresponding to the current operation action.

Description

Technical Field

[0001] This application relates to the field of robot systems, and more particularly to a robot system and its control method and medium. Background Technology

[0002] Robotic systems typically provide at least one function and a function setting method for users to configure at least one function. Existing function setting methods include: one through a menu interface, where users must remain engaged with the input device and find it difficult to interact with other devices; another method involves activating foot pedals, but this increases the number of foot pedals required when multiple functions are available, and the ease of use of the function setting method significantly impacts the convenience of interaction. Summary of the Invention

[0003] In view of this, this application provides a robot system and its control method and medium.

[0004] In a first aspect, this application provides a robot system, the robot system comprising: an output device for outputting instruction information, the instruction information including at least one operation action instruction, the at least one operation action instruction corresponding one-to-one with at least one operation action on an input device, the at least one operation action corresponding one-to-one with at least one function of the robot system; the input device, configurable to a follow mode or a controlled mode, wherein when configured to follow mode, it is coupled to a manipulator component to cause the manipulator component to follow the movement of the input device, and when configured to controlled mode, it is decoupled from the manipulator component and can execute any operation action indicated by the operation action instruction; and a control device for detecting the current operation action performed on the input device, determining the function corresponding to the current operation action, and controlling the robot system to activate the function corresponding to the current operation action when the input device is configured to controlled mode.

[0005] Secondly, this application provides a control method for a robot system. The robot system includes an input device and an output device. The output device is used to output indication information, which includes at least one operation action indication. The at least one operation action indication corresponds one-to-one with at least one operation action on the input device. The at least one operation action corresponds one-to-one with at least one function of the robot system. The input device can be configured to a follow mode or a controlled mode. When configured to follow mode, it is coupled to a manipulator component to make the manipulator component follow the movement of the input device. When configured to controlled mode, it is decoupled from the manipulator component and can execute the operation action indicated by any one of the operation action indications. The method includes: when the input device is configured to the controlled mode, detecting the current operation action performed on the input device; determining the function corresponding to the current operation action; and controlling the robot system to activate the function corresponding to the current operation action.

[0006] Thirdly, this application provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the steps of the method described in the second aspect.

[0007] In this embodiment, the output device outputs an operation action indication to indicate the executable operation. When the input device in the robot system is configured in controlled mode, it is decoupled from the manipulator component and can execute the operation action indicated by the operation action indication. This allows the control device to determine the function corresponding to the current operation action and then control the robot system to activate the function corresponding to the current operation action. This allows the user to hold the input device without having to remove their hand to perform other operations while setting up the function, allowing them to focus on setting up the function. It also makes the function setting more intuitive and easy to use, reduces the probability of errors, and improves the user experience.

[0008] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0009] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0010] Figure 1 This is a schematic diagram of the structure of a surgical robot system shown in an exemplary embodiment of this application;

[0011] Figure 2 This is a schematic diagram of the structure of a console shown in an exemplary embodiment of this application;

[0012] Figure 3a, 3b This is a schematic diagram of the structure of a robot manipulation device shown in an exemplary embodiment of this application;

[0013] Figure 4a , Figure 4b , Figure 4c and Figure 4d These are schematic diagrams illustrating visual instruction information in an exemplary embodiment of this application;

[0014] Figure 5a This is a schematic diagram illustrating a visual instruction information according to an exemplary embodiment of this application;

[0015] Figure 5b yes Figure 5a A schematic diagram of the operational action instructions in the visual instructions shown;

[0016] Figure 6a This is a schematic diagram illustrating another visual indication as shown in an exemplary embodiment of this application;

[0017] Figure 6b yes Figure 6a A schematic diagram of the operational action instructions in the visual instructions shown;

[0018] Figure 7a This is a schematic diagram illustrating another visual indication as shown in an exemplary embodiment of this application;

[0019] Figure 7b yes Figure 7a A schematic diagram of the operational action instructions in the visual instructions shown;

[0020] Figure 8a and Figure 8b These are schematic diagrams illustrating progress indications in an exemplary embodiment of this application;

[0021] Figure 9 This is a flowchart illustrating a control method for a robot system according to an exemplary embodiment of this application. Detailed Implementation

[0022] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0023] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0024] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0025] Surgical robotic systems are advanced medical systems designed to help surgeons perform more precise and safer surgical procedures. Please see [link / reference]. Figure 1 The surgical robot system 10 includes one or more consoles 101, a robot manipulator 102, and a control device 103. The consoles 101 can control the robot manipulator 102 to follow inputs from the consoles 101, thereby performing surgery on tissue at the surgical site. The robot manipulator 102 can acquire images of the surgical site using its imaging device and transmit them to the consoles 101. The consoles 101 then provide the user with images of the surgical site, allowing the user to view these images and assisting in surgical operations via the consoles 101. The control device 103 includes at least one memory and at least one processor, and is communicatively connected to the consoles 101 and the robot manipulator 102, enabling control between the consoles 101 and the robot manipulator 102. The control device 103 can execute several programmed instructions (e.g., a computer-readable medium storing the instructions) via the processor to implement some or all of the methods described in the embodiments of this application. Physically, the control device 103 can also be integrated into the console 101 or the robot manipulation device 102, or it can be a stand-alone computer device. This application embodiment does not impose specific limitations on the structural form of the control device 103. In addition, the surgical robot system 10 may also include an auxiliary device 104 (not shown in the figure). The auxiliary device 104 can provide the surgical site image to the assistant or other assistive physician by receiving the surgical site image acquired by the robot manipulation device 102.

[0026] like Figure 2As shown, the console 101 includes an input device 1011 and an output device 1012. The input device 1011 can be coupled to the manipulator assembly 1021 on the robot manipulation device 102, and the pose of the manipulator assembly 1021 can be adjusted by controlling the input device 1011. The input device 1011 may include a first input device 1011a (also known as a left master hand controller) and a second input device 1011b (also known as a right master hand controller), which can be operated by the user with their left and right hands respectively. The output device 1012 can output an image of the surgical site captured by the imaging device, which can be a stereoscopic image. The output device 1012 can be a stereoscopic vision display with a left eyepiece 1013 and a right eyepiece 1014. Furthermore, the console 101 may also include foot pedals 1015a and 1015b for various functional operations and human-machine interaction.

[0027] like Figure 3a The diagram illustrates a structural example of a robot manipulation device 102. This robot manipulation device 102 is a single-port robot manipulation device, comprising a manipulator assembly 1021. This assembly 1021 can be a manipulator arm capable of mounting multiple surgical tools 20, or a detachable manipulator arm with multiple surgical tools 20 mounted on it. The manipulator arm includes several connecting arms, with adjacent connecting arms 1021a moving relative to each other with specific degrees of freedom, allowing the end effector of the manipulator arm to achieve multiple degrees of freedom (e.g., 7 degrees of freedom, varying depending on the surgical tools mounted on it). The surgical tools 20 mounted on the manipulator arm may include surgical instruments or imaging devices. The imaging device can image the surgical site and may include, but is not limited to, endoscopes, optical cameras, and ultrasound probes. Surgical instruments can perform surgical manipulations on the tissue at the surgical site; such instruments may include, but are not limited to, electrocautery devices, clamps, and vascular occluders. The surgical instruments can be inserted into the patient's body through the guide 50. Figure 3bAs shown, the robot manipulation device 102 includes at least one manipulator assembly 1021, which can be a manipulator arm or a detachable manipulator arm with surgical tools 20 mounted on it. The manipulator arm includes several connecting arms, with adjacent connecting arms moving relative to each other with specific degrees of freedom, allowing the end effector of the manipulator arm to achieve multiple degrees of freedom (e.g., 7 degrees of freedom, depending on the surgical tools mounted on it). The surgical tools 20 mounted on the manipulator arm can include surgical instruments or imaging devices. The imaging device can image the surgical site and may include, but is not limited to, endoscopes, optical cameras, ultrasound probes, etc. The surgical instruments can perform surgical operations on the tissue at the surgical site, such as, but not limited to, electrocautery devices, clamps, vascular occluders, etc.

[0028] The surgical robot system 10 typically provides at least one function and function settings to allow users to switch between multiple functions. Under different functions, the input device 1011 can interact with different objects in the surgical robot system 10. These objects can be various manipulator components 1021 of the surgical robot system 10 or the user interface of the surgical robot system 10, etc.

[0029] In different operating modes, the input device 1011 can be coupled to different manipulator components 1021 to control the position and orientation of the surgical instruments 20 on the different manipulator components 1021. Alternatively, it can be uncoupled from the manipulator components 1021 and interact with the user interface of the surgical robot system 10. For example, the robot system 10 has the following three operating modes: "tool tracking" operating mode, "imaging device" operating mode, and "mouse" operating mode. In the "tool tracking" operating mode, the input device 1011 is coupled to the manipulator arm on which the surgical instruments are mounted, and the movement of the surgical instruments can be controlled through the input device 1011. In the "imaging device" operating mode, the input device 1011 is coupled to the manipulator arm on which the imaging device is mounted, and the movement of the imaging device can be controlled through the input device 1011. In the "mouse" operating mode, the input device 1011 is simulated as a mouse, and the user interface output by the output device 1012 can be interacted through the input device 1011. Among the existing function setting methods, there is one method that involves interacting with the menu interface, which requires the user to keep using the input device and makes it difficult to interact with other devices; another method is to set up functions by activating foot pedals, but when there are multiple functions, the number of foot pedals required also increases, and the ease and speed of setting up functions greatly affects the convenience of interaction.

[0030] Based on this, embodiments of this application provide a robot system 10 and its control method and medium.

[0031] In some embodiments, please refer to Figure 1 , Figure 2 , Figure 3a and Figure 3b The manipulator component 1021 in the robot system 10 provided in this application embodiment can be coupled or decoupled from the input device 1011. When coupled to the input device 1011, the manipulator component 1021 can follow the movement of the input device 1011. The input device 1011 can be configured to follow mode or controlled mode. When configured to follow mode, it is coupled to the manipulator component 1021 so that the manipulator component 1021 follows the movement of the input device 1011. When configured to controlled mode, it is decoupled from the manipulator component 1021 and can execute the operation action indicated by the operation action instruction. This allows the user to hold the input device without taking their hand out to perform other operations during the function setting process, allowing them to focus on function setting. It also makes function setting more intuitive and easy to use, reduces the probability of errors, and improves the user experience.

[0032] It should also be noted that before being configured into controlled mode, input device 1011 can be either decoupled from or coupled to manipulator component 1021. For example, in the current operating mode of robot system 10, input device 1011 is decoupled from manipulator component 1021. In this case, when input device 1011 is configured into controlled mode, it remains decoupled from manipulator component 1021. The user can execute the operation action indicated by the operation action instruction through input device 1011. After control device 103 determines the function corresponding to the current operation action, it controls robot system 10 to activate the function corresponding to the current operation action. For example, in the current operating mode of the robot system 10, the input device 1011 is coupled to the first manipulator component 1021. In this case, when the input device 1011 is configured to the controlled mode, the input device 1011 is decoupled from the first manipulator component 1021. The user can execute the operation action indicated by the operation action instruction through the input device 1011. After the control device 103 determines the function corresponding to the current operation action, it controls the robot system 10 to activate the function corresponding to the current operation action.

[0033] It should be understood that the control device 103 is communicatively connected to the input device 1011. When the input device 1011 is configured in controlled mode, the control device 103 can detect the current operation of the input device 1011, determine the function corresponding to the current operation, and then control the robot system 10 to activate the function corresponding to the current operation.

[0034] In some embodiments, the output device 1012 in the robot system 10 outputs instruction information to guide the user to perform function setting operations via the input device 1011. Specifically, the instruction information may indicate at least one of the following:

[0035] (1) Operate the input device 1011 in a preset direction. For example, the instruction information may indicate to operate the input device 1011 to the left to activate function A, or to operate the input device 1011 to the right to activate function B, etc.

[0036] (2) The input device 1011 is operated using a preset duration. For example, the instruction information can instruct to press and hold the input device 1011 to activate function A, press and hold the input device 1011 to activate function B, etc.

[0037] (3) The input device 1011 is operated using a preset number of operations. For example, the instruction information may instruct the input device 1011 to perform one operation to activate function A, or the input device 1011 to perform two operations to activate function B, etc.

[0038] (4) Operate a preset number of input devices 1011 among the multiple input devices 1011. For example, the instruction information may instruct to operate one input device 1011 to activate function A, or to operate two input devices 1011 to activate function B, etc.

[0039] (5) Operate multiple input devices 1011 in a preset order. For example, the instruction information may instruct to operate the left input device 1011 first and then the right input device 1011 to activate function A, or to operate the right input device 1011 first and then the left input device 1011 to activate function B, etc.

[0040] (6) Operate a specified input device among the multiple input devices 1011. For example, the instruction information may instruct to operate the input device 1011 on the left to activate function A, and to operate the input device 1011 on the right to activate function B, etc.

[0041] In this embodiment, by outputting instruction information through output device 1012, the user can be guided to perform function setting operations through input device 1011. The function settings are more intuitive and easier to use, improving the user experience. For example, output device 1012 is communicatively connected to control device 103. When input device 1011 is configured in controlled mode, control device 103 sends relevant control commands to output device 1012 to cause output device 1012 to output instruction information.

[0042] The output device 1012 may include, but is not limited to, at least one of the following: a visual display, an audio output device, a vibration output device, or a movable component. Correspondingly, the indication information may include, but is not limited to, at least one of the following: visual indication information, audio indication information, vibration indication information, or motion indication information. Examples of various types of output devices 1012 and indication information are described below.

[0043] In a specific embodiment, such as Figure 2 As shown, the output device 1012 is a visual display on the console 101, and the indication information is visual indication information 40. This visual indication information 40 provides action guidance for the user to perform function setting operations via the input device 1011. For example, the visual indication information 40 includes character indication information. Figure 4a As shown, the visual instruction information 40 is displayed as text on the surgical screen 30. The visual instruction information 40 is "Move left to activate function A", which means that the user can move the input device 1011 to the left to set the robot system 10 to function A.

[0044] For example, visual indication information 40 includes menu indication information. (e.g.) Figure 4b As shown, visual instruction information 40 is displayed on the surgical screen 30 in the form of one or more menu items. Visual instruction information 40 includes "move left to activate function A" and "move right to activate function B", which correspond to different menu items, indicating that the user can move the input device 1011 to the left to set the robot system 10 to function A, and can move the input device 1011 to the right to set the robot system 10 to function B.

[0045] For example, the visual indication information 40 includes graphic indication information. This graphic indication information can be arrow indication information, such as... Figure 4c As shown, visual indication information 40 is displayed on the surgical screen 30 as an arrow. Visual indication information 40 includes a left-pointing arrow, corresponding to function A, indicating that the user can move the input device 1011 to the left to set the robot system 10 to function A. Alternatively, the graphical indication information can be an icon corresponding to the function, such as... Figure 4d As shown, the visual instruction information 40 is displayed on the surgical screen 30 in the form of function icons. The visual instruction information 40 includes the icon X corresponding to the current function and the icon Y corresponding to function A, indicating that the user can move the input device 1011 to the left to set the robot system 10 from the current function to function A.

[0046] For example, the visual instruction information 40 includes animated instruction information, which may be an operation tutorial. For instance, in the example of moving the robot system 10 to the left by manipulating the input device 1011 to switch the robot system 10 from its current function to function A, the animated instruction information may be an animation of moving the input device 1011 to the left.

[0047] The visual instruction information 40 may also include at least two of the above-mentioned instruction information to make the visual instruction information 40 more vivid and easier to understand. For example, it may include both arrow instruction information and character instruction information, or it may include both icon instruction information, arrow instruction information, and character instruction information.

[0048] In one specific embodiment, the output device 1012 is an audio output device, and the indication information is audio indication information. For example, the voice indication information includes a prompt such as "Move the manipulation input device 1011 to the left to activate function A".

[0049] In one specific embodiment, the output device 1012 is a vibration output device, and the indication information is vibration indication information. For example, when the output device 1012 outputs vibration indication information, the input device 1011 can be manipulated to activate function A; when the output device 1012 does not output vibration indication information, the input device 1011 can be manipulated to activate function B. Alternatively, when the output device 1012 outputs vibration indication information of a longer duration, the input device 1011 can be manipulated to activate function A; when the output device 1012 outputs vibration indication information of a shorter duration, the input device 1011 can be manipulated to activate function B.

[0050] In one specific embodiment, the output device 1012 is a movable component, and the indication information is motion indication information. For example, when the movable component moves to a certain position, the input device 1011 can be manipulated to activate function A, and when the movable component moves to another position, the input device 1011 can be manipulated to activate function B.

[0051] In practical applications, the output device 1012 may include at least two of the types described above, and correspondingly, the indication information may also include at least two of the types described above. For example, the output device 1012 may include a visual display and an audio output device, and the indication information may include visual indication information and audio indication information. Alternatively, the output device 1012 may include an audio output device and a vibration output device, and the indication information may include audio indication information and vibration indication information.

[0052] It should be noted that, in addition to the operation action instructions described in the above embodiments, the operation action instructions in the instruction information can also be represented in other forms, as long as they can guide the user to perform the corresponding operation action through the input device 1011. In addition, the operation action instructions can be in addition to guiding the user to perform translational actions on a two-dimensional plane through the input device 1011, they can also be operation actions on a three-dimensional plane. In this regard, the embodiments of this application do not impose specific limitations.

[0053] In some embodiments, the indication information includes at least one operation action indication, which corresponds one-to-one with at least one operation action performed on the input device 1011, and the at least one operation action corresponds one-to-one with at least one function of the robot system 10. For example, when the indication information is arrow indication information, the at least one operation action indication includes multiple arrows pointing in different directions, and the at least one operation action includes an operation action of operating the input device 1011 along at least one operation direction, wherein the at least one operation direction corresponds one-to-one with the direction of the at least one arrow.

[0054] The functions of the robot system 10 include, but are not limited to, at least one of the following:

[0055] (1) Function to select the operating mode of robot system 10. For example, the operating mode of robot system 10 can be set to "tool tracking" operating mode, "imaging device" operating mode or "mouse" operating mode by operating input device 1011.

[0056] (2) A function to adjust the image displayed by the robot system 10. For example, if the image displayed by the robot system 10 is acquired by multiple vision sensors, the robot system 10 can be controlled to display the image acquired by at least one of the multiple vision sensors by operating the input device 1011.

[0057] (3) The function of adjusting the parameters of the robot system 10. The above parameters include, but are not limited to, at least one of the following parameters of the robot system 10: display parameters (e.g., resolution, frame rate, brightness, contrast, viewing angle, etc.), kinematic parameters, sensor parameters, power system parameters, communication parameters, safety parameters, damping parameters, etc.

[0058] (4) Function to adjust the state of components on robot system 10. For example, to adjust the motion state of manipulator component 1021, display state of display component, communication state of communication component, etc.

[0059] It is understood that if the robot system 10 can currently perform more than one function, the instruction information may include a corresponding number of operation action instructions. Each operation action instruction is bound to a function specified in the robot system 10 and is used to guide the user to perform the corresponding operation action through the input device 1011 to set the robot system 10 to the corresponding function.

[0060] For example, such as Figure 5a and Figure 5b As shown, the robot system 10 can currently be set to functions A and B. In the visual instruction information 40, two arrows can be used to represent two operation action instructions 401. The left arrow is operation action instruction 401a, which means "move left". When the user performs the "move left" operation action through the input device 1011, the control device 103 sets the robot system 10 to function A. The right arrow is operation action instruction 401b, which means "move right". When the user performs the "move right" operation action through the input device 1011, the control device 103 sets the robot system 10 to function B.

[0061] For example, such as Figure 6a and Figure 6b As shown, the robot system 10 currently has four configurable functions: function A, function B, function C, and function D. In the visual instruction information 40, four arrow directions represent four operation action instructions 401. The upward arrow is operation action instruction 401a, indicating "move upward." When the user performs the "move upward" operation through the input device 1011, the control device 103 switches the robot system 10 to function A. The right arrow is operation action instruction 401b, indicating "move right." When the user performs the "move right" operation through the input device 101... When the user performs the "move right" operation, the control device 103 switches the robot system 10 to function B, with the down arrow indicating operation action 401c, which means "move down". When the user performs the "move down" operation through the input device 1011, the control device 103 switches the robot system 10 to function C, with the left arrow indicating operation action 401d, which means "move left". When the user performs the "move left" operation through the input device 1011, the control device 103 switches the robot system 10 to function D.

[0062] For example, such as Figure 7a and Figure 7bAs shown, the robot system 10 can currently be set to functions A to H. In the visual instruction information 40, eight arrow directions can be used to represent eight operation action instructions 401A to 401H. When the user executes any one of the operation action instructions 401a to 401h through the input device 1011, the control device 103 sets the robot system 10 to the function corresponding to that operation action instruction.

[0063] In one specific embodiment, the input device 1011 has an operating space with multiple degrees of freedom in three-dimensional space, and can perform operating actions in three-dimensional space. The operating action indication is an action indication in two-dimensional plane. There is a pre-established correspondence between the action indication in two-dimensional plane and the operating actions in three-dimensional space of the input device 1011, thereby utilizing the multi-degree-of-freedom operating advantage of the input device 1011 to provide multi-directional function setting operations.

[0064] It should be noted that the operation action instructions included in the instruction information can be fixed or dynamically changing. In an embodiment where the operation action instructions change dynamically, the type of operation action instruction can be determined based on at least one of the following: historical operation actions of the input device 1011; the state of the robot system 10; and the historical operation duration of the input device 1011. In this way, different functions of the robot system 10 can be activated under different circumstances through different types of operation action instructions.

[0065] Since different historical operation actions may trigger the robot system 10 to execute different functions or enter different states, the type of operation action indication can be determined based on the historical operation actions performed on the input device 1011. For example, when the historical operation action on the input device 1011 is to move the input device 1011 to the left, the robot system 10 is triggered to activate function A; when the historical operation action on the input device 1011 is to move the input device 1011 to the right, the robot system 10 is triggered to activate function B. Assuming that when function A is activated, the robot system 10 can be triggered to activate function C, and when function B is activated, the robot system 10 can be triggered to activate function D, then when the historical operation action on the input device 1011 is to move the input device 1011 to the left, the type of operation action indication included in the indication information can include the type corresponding to function C (e.g., moving the input device 1011 upwards); when the historical operation action on the input device 1011 is to move the input device 1011 to the right, the type of operation action indication included in the indication information can be the type corresponding to function D (e.g., moving the input device 1011 downwards).

[0066] The functions that can be triggered may differ depending on the state of the robot system 10. For example, in the power-on state, user interface preparation functions, environmental perception functions, and / or self-test and system diagnostic functions can be triggered; in the operation state, motion control functions and visual feedback functions can be triggered; and in the fault state, alarm functions and emergency stop functions can be triggered. Therefore, the type of operation action indication can be determined based on the state of the robot system 10. For example, in the power-on state, the type of operation action indication may include the type corresponding to the user interface preparation function (e.g., single operation input device 1011) and the indication corresponding to the environmental perception function (e.g., multiple operation input device 1011); in the fault state, the type of operation action indication may include the type corresponding to the alarm function (e.g., long press input device 1011).

[0067] Different historical operation durations on input device 1011 may indicate different functions that the user expects to trigger. For example, a long press on input device 1011 indicates an expectation to trigger function A, while a short press indicates an expectation to trigger either function B or function C. Therefore, the type of operation action indication can be determined based on the historical operation duration on input device 1011. For example, when the historical operation duration on input device 1011 is a long press, the type of operation action indication includes the type corresponding to function A (e.g., operating input device 1011 on the left), and when the historical operation duration on input device 1011 is a short press, the type of operation action indication includes the type corresponding to function B (e.g., operating input device 1011 in the middle) and the type corresponding to function C (e.g., operating input device 1011 on the right).

[0068] In some embodiments, the output device 1012 is further configured to output a progress indicator, which indicates the progress of the robot system 10 in executing the function corresponding to the current operation. Specifically, the control device receives a three-dimensional movement command from the input device 1011, and then calculates the movement information projected onto the two-dimensional plane based on the three-dimensional motion form of the input device 1011, and updates and displays the progress indicator in real time based on the movement information.

[0069] For example, the progress indicator 402 uses the degree of fill inside the arrow to indicate the progress of the robot system 10 in executing the function corresponding to the current operation. For example... Figure 8a As shown, the arrow is not yet fully filled, indicating that the robot system 10 has not yet completed the execution of the function corresponding to the current operation action; as Figure 8bAs shown, the arrow is completely filled, indicating that the robot system 10 has completed the execution of the function corresponding to the current operation. It should be noted that the progress indication can also be expressed by the length of the arrow to indicate the execution progress of the robot system 10 to the current operation, or by voice or other means. For example, the output device 1012 can output a voice progress indication similar to "Function A is 80% complete". This application embodiment does not impose specific limitations on the form of the progress indication.

[0070] In some embodiments, the control device 103 is further configured to configure the input device 1011 into a controlled mode in response to a controlled mode switching activation command.

[0071] It should be understood that the control device 103 is connected to the input device 1011. When the control device 103 receives a controlled mode switching activation command, it activates the mode switching and configures the input device 1011 into the controlled mode.

[0072] In one specific embodiment, the controlled mode switching activation command is triggered by the user through the activation device of the robot system 10.

[0073] For example, the activation device may include a button, which the user can press to issue a controlled mode switching activation command. Or, as... Figure 2 As shown, the user triggers the controlled mode switching activation command via the foot pedals 1015a and 1015b on the console 101. The user can also trigger the controlled mode switching activation command via the touchscreen output by the output device 1012. Furthermore, the controlled mode switching command can also be triggered by the user via voice control, or via a joystick, knob, slider, etc. This embodiment does not specifically limit the method of triggering the controlled mode switching activation command.

[0074] In some embodiments, the control device 103 is further configured to respond to a controlled mode switching deactivation command to keep the robot system 10 in the operating mode prior to the activated controlled mode switching.

[0075] For example, robot system 10 is currently in operation mode A. In this case, if control device 103 responds to the controlled mode switching activation command, it activates the controlled mode switching of robot system 10 and configures input device 1011 to the controlled mode. If control device 103 receives a controlled mode switching deactivation command afterward, it deactivates the controlled mode switching of robot system 10 and keeps robot system 10 in the operation mode before the controlled mode switching was activated, i.e., operation mode A.

[0076] Please see Figure 9 , Figure 9A flowchart illustrating the control method of the robot system 10 provided in an embodiment of this application is shown. For ease of understanding, it is assumed that the currently set function is used to switch the operating mode of the robot system 10. Figure 9 As shown, the method includes steps S910 to S970.

[0077] In step S910, the robot system 10 is in the current operating mode. For example, the current operating mode can be a common operating mode of the robot system 10, such as the "tool tracking" operating mode.

[0078] In step S920, the method determines whether the controlled mode switching is activated, that is, whether the control device 103 receives a controlled mode activation command. For example, the user can trigger the controlled mode activation command through a foot pedal on the console, a button on the input device, a user interface, or a voice command. If the determination result in step S920 is "no", the method loops back to step S910 and controls the robot system 10 to remain in the current operating mode. If the determination result in step S920 is "yes", the process proceeds to step S930.

[0079] In step S930, the method configures the input device into a controlled mode. When the input device 1011 is configured into a controlled mode, it is decoupled from the manipulator component 1021 and can perform function setting operations.

[0080] In step S940, the method determines whether the controlled mode switching is activated. When the control device 103 receives the controlled mode switching deactivation command, it determines that the controlled mode switching is deactivated. If the judgment result in step S940 is "no", the method loops back to step S910 and controls the robot system 10 to remain in the current operation mode. If the judgment result in step S940 is "yes", the process proceeds to step S950.

[0081] In step S950, the method waits and determines whether the input device 1011 performs the operation action indicated by the operation action instruction. If the determination result of step S950 is "no", the method loops back to step S920 and determines whether the controlled mode switching is activated. If the controlled mode switching is continuously activated, the input device 1011 can remain in the current controlled mode. If the determination result in step S950 is "yes", the method proceeds to step S960.

[0082] In step S960, the control device determines the function corresponding to the current operation action and controls the robot system 10 to be set to the function corresponding to the current operation action.

[0083] In step S970, the method determines whether the controlled mode switching is activated. When the control device receives the controlled mode switching deactivation command, it determines that the controlled mode switching is deactivated. If the judgment result in step S970 is "no", the method loops back to step S910 and controls the robot system 10 to remain in the current operating mode. If the judgment result in step S940 is "yes", step S960 is maintained and the robot system 10 is controlled to remain in the controlled mode.

[0084] In some embodiments, when the input device 1011 is configured in a controlled mode, the control device 103 is also configured to provide first tactile feedback to the input device 1011 to limit the input device 1011 from performing any movement other than at least one operational action.

[0085] Specifically, the control device 103 calculates the travel limit of the input device 1011 in three-dimensional space based on the operation action of the input device 1011 used for function setting, and then applies tactile force feedback to the input device 1011, making it easier for the user to understand the operation method of function setting, ensuring that the user can input the correct operation action, and reducing the probability of incorrect operation.

[0086] For example, the predefined operation actions in the robot system 10 include two specific directions: moving left and moving right. In this case, the control device provides first tactile feedback to the input device 1011 to restrict the input device 1011 from performing movements other than the two specific directions of moving left and moving right. When the user performs an operation other than the permitted action on the input device 1011, the input device 1011 will apply tactile force feedback to guide the user back to its permitted position.

[0087] In some embodiments, the control device 103 is further configured to determine the function corresponding to the current operation action when it is determined that the current operation action performed by the input device 1011 has reached a travel threshold.

[0088] It should be understood that when the input device 1011 slides to a specified travel distance with a specific operation action, that is, when the current operation action performed by the input device 1011 reaches the travel distance threshold, the control device 103 determines the function corresponding to the current operation action. For example, the travel distance threshold may be 50%, 60%, or 70% of the movable travel distance of the input device 1011, etc., and can be adjusted according to actual needs. This application embodiment does not impose specific limitations on this.

[0089] In this embodiment, by setting a travel threshold and determining the function corresponding to the current operation when the current operation performed by the input device 1011 reaches the travel threshold, the current operation performed by the input device 1011 does not need to reach the entire movable travel of the input device 1011, so that the user can quickly set the functions of the robot system 10.

[0090] In some embodiments, the control device 103 is further configured to provide a second tactile feedback to the input device 1011 to indicate how close the current operation performed by the input device 1011 is to the aforementioned travel threshold.

[0091] This embodiment uses haptic feedback to indicate to the user how close their current operation is to the aforementioned travel threshold, allowing the user to more intuitively experience the function setting process. Specifically, the control device 103 receives a three-dimensional movement command from the input device 1011, then calculates the movement information projected onto the two-dimensional plane based on the three-dimensional motion of the input device 1011, and applies haptic feedback based on this movement information and the travel threshold.

[0092] In some embodiments, the intensity of the second tactile feedback is positively correlated with the proximity.

[0093] It should be understood that as the user performs an operation through the input device 1011, the closer the operation is to the target location, the stronger the tactile feedback of the input device 1011 becomes, in order to prompt the user that the current operation is about to trigger a function setting and improve the user experience.

[0094] In one specific embodiment, the feedback frequency of the second tactile feedback is also positively correlated with the degree of proximity. The closer the current operation performed by the input device 1011 is to the aforementioned travel threshold, the higher the tactile feedback frequency of the input device 1011.

[0095] In some embodiments, the robot system 10 includes a plurality of input devices 1011, and the control device 103 can configure at least one input device 1011 into a controlled mode.

[0096] For example, such as Figure 2As shown, the robot system 10 includes a first input device 1011a and a second input device 1011b. The current operating mode of the robot system 10 is the "tool tracking" operating mode. In this operating mode, the first input device 1011a is coupled to the first manipulator component in the robot system 10, and the second input device 1011b is coupled to the second manipulator component in the robot system 10. When the control device 103 receives a controlled mode switching activation command, the control device 103 configures the second input device 1011b into a controlled mode. In the controlled mode, the second input device 1011b is decoupled from the second manipulator component and can execute the operation action indicated by the operation action instruction. For the first input device 1011a, in one case, the first input device 1011a can continue to be coupled to the first manipulator component, and the first manipulator component moves with the first input device 1011a. In another case, the first manipulator component coupled to the first input device 1011a is soft-locked, so that the first manipulator component does not move with the first input device 1011a, thus avoiding misoperation during mode switching. In addition, when the control device 103 receives the controlled mode switching activation command, the control device 103 can also select to configure the first input device 1011a as the controlled mode, or configure both the first input device 1011a and the second input device 1011b as the controlled mode, and perform the function settings of the robot system 10 according to the controlled mode switching operation performed by the first input device 1011a and the second input device 1011b.

[0097] In some embodiments, the control device 103 is further configured to determine whether the current operation action matches the operation action indicated by at least one operation action indicator, and if it is determined that the current operation action matches the operation action indicated by a preset operation action indicator, configure the robot system 10 to function corresponding to the operation action indicated by the preset operation action indicator.

[0098] It is understood that the robot system 10 defines at least one operation action indicated by an operation action indicator. The operation action indicated by the at least one operation action indicator is the operation action performed by the input device 1011 when setting the function of the robot system 10. In response to detecting the operation action performed by the input device 1011, the control device 103 determines whether the current operation action matches the operation action indicated by the at least one operation action indicator, that is, whether the user operates the input device 1011 according to the predefined operation action. If it is determined that the current operation action matches the operation action indicated by the preset operation action indicator, the robot system 10 can be set to the corresponding function.

[0099] For example, taking the above-mentioned function settings for switching the operation mode of the robot system 10 as an example, assuming that the current operation mode of the robot system 10 is the "imaging device" operation mode, the robot system 10 currently provides a switchable operation mode: the "tool tracking" operation mode. Correspondingly, the robot system 10 defines a preset action "move left". This preset action is the operation action performed by the input device 1011 to switch to the "tool tracking" operation mode. Only when the user performs the "move left" mode switching operation through the input device 1011 will the control device 103 control the robot system 10 to switch to the "tool tracking" operation mode.

[0100] For example, the robot system 10 currently provides four switchable operating modes: operating mode A, operating mode B, operating mode C, and operating mode D. Correspondingly, the robot system 10 defines a preset action "move left". The control device 103 responds to the mode switching operation performed by the input device 1011, and controls the robot system 10 to switch to operating mode A when it is determined that the mode switching operation matches the preset action "move left". If the user continues to perform the "move left" operation through the input device 1011, the control device 10 continues to switch to the next operating mode, i.e., operating mode B, and so on.

[0101] It should also be noted that, in addition to the above-mentioned function settings, the control device 103 can also directly control the corresponding function activation of the robot system 10 in response to the operation action performed by the input device 1011. That is, when the input device 1011 is configured in controlled mode, the control device 103 will control the corresponding function activation of the robot system 10 when the user performs any operation action through the input device 1011.

[0102] In some embodiments, the control device 103 is further configured to determine whether the current operation action matches multiple different preset actions, and if it is determined that the current operation action matches one of the preset actions, control the robot system 10 to activate the function corresponding to the preset action, wherein each preset action corresponds to a function specified in the robot system 10.

[0103] For example, the robot system 10 currently provides four configurable functions: function A, function B, function C, and function D. Functions A, B, C, and D correspond to preset actions "move up", "move right", "move down", and "move left", respectively. The control device 103 responds to the current operation performed by the input device 1011 and determines whether the current operation matches the preset actions "move up", "move right", "move down", and "move left". For example, if it is determined that the current operation matches the preset action "move down", then function C corresponding to the preset action "move down" is activated. That is, when the user controls the input device 1011 to move down, the robot system 10 is controlled to activate function C. Or, if it is determined that the current operation matches the preset action "move left", then function D corresponding to the preset action "move left" is activated. That is, when the user controls the input device 1011 to move left, the robot system 10 is controlled to activate function D.

[0104] It is easy to understand that the solutions described in the above embodiments can be combined when there is no conflict, and not all of them are listed in the embodiments of this application.

[0105] like Figure 1 As shown in Figure 3, this application embodiment also provides a control method for a robot system. The robot system 10 includes an input device 1011 and an output device 1012. The output device 1012 is used to output indication information, which includes at least one operation action indication. The at least one operation action indication corresponds one-to-one with at least one operation action on the input device 1011, and the at least one operation action corresponds one-to-one with at least one function of the robot system 10. The input device 1011 can be configured to a follow mode or a controlled mode. When configured to follow mode, it is coupled to a manipulator component 1021 so that the manipulator component 1021 follows the movement of the input device 1011. When configured to controlled mode, it is decoupled from the manipulator component 1021 and can execute any operation action indicated by the operation action indication. The method includes:

[0106] When the input device 1011 is configured in controlled mode, the current operation performed on the input device 1011 is detected;

[0107] Determine the function corresponding to the current operation action, and control the robot system 10 to activate the function corresponding to the current operation action.

[0108] In the function activation method of the robot system provided in this application embodiment, when the input device 1011 of the robot system 10 is configured in controlled mode, it is decoupled from the manipulator component 1021 and can execute the operation action indicated by the operation action instruction, so that the control device 103 determines the function corresponding to the current operation action, and then controls the robot system 10 to activate the function corresponding to the current operation action. This allows the user to hold the input device 1011 without taking their hand out to perform other operations during the function setting process, so as to focus on the function setting. It also makes the function setting more intuitive and easy to use, reduces the probability of errors, and improves the user experience.

[0109] In some embodiments, the type of indication information includes at least one of the following: audio indication information, visual indication information, vibration indication information, and motion indication information.

[0110] In some embodiments, the visual indication information includes at least one of the following: graphic indication information, menu indication information, character indication information, color indication information, and animation indication information.

[0111] In some embodiments, the graphic indication information is arrow indication information, at least one operation action indication includes multiple arrows pointing in different directions, at least one operation action includes an operation action of operating the input device 1011 along at least one operation direction, wherein at least one operation direction corresponds one-to-one with the direction of at least one arrow.

[0112] In some embodiments, the indication information is used to indicate at least one of the following: operating the input device in a preset direction; operating the input device for a preset duration; operating the input device for a preset number of operations; operating a preset number of input devices out of a plurality of input devices; operating a plurality of input devices in a preset order.

[0113] In some embodiments, the functions of the robot system 10 include at least one of the following: the function of selecting the operating mode of the robot system 10; the function of adjusting the image displayed by the robot system 10; the function of adjusting the parameters of the robot system 10; and the function of adjusting the state of the components on the robot system 10.

[0114] In some embodiments, the method further includes: configuring the input device 1011 into a controlled mode in response to a controlled mode activation command.

[0115] In some embodiments, the robot system 10 further includes an activation device, through which a controlled mode activation command is triggered.

[0116] In some embodiments, the activation device includes at least one of the following: a foot pedal, a button, a joystick, a touch screen, a knob, and a slider.

[0117] In some embodiments, the method further includes providing first haptic feedback to the input device 1011 to restrict the input device 1011 from performing any movement other than at least one operational action.

[0118] In some embodiments, determining the function corresponding to the current operation action includes: determining the function corresponding to the current operation action when it is determined that the current operation action performed by the input device 1011 has reached a travel threshold.

[0119] In some embodiments, the method further includes providing a second haptic feedback to the input device 1011 to indicate how close the current operation performed by the input device 1011 is to a travel threshold.

[0120] In some embodiments, the intensity of the second tactile feedback is positively correlated with the proximity.

[0121] In some embodiments, the robot system 10 includes a plurality of input devices 1011, and the method further includes configuring at least one input device 1011 into a controlled mode.

[0122] In some embodiments, the robot system 10 includes a plurality of input devices 1011, the plurality of input devices 1011 including a first input device configured in a follow mode and a second input device configured in a controlled mode; the method further includes: controlling the first input device to be in a locked state when the second input device is operated.

[0123] In some embodiments, the method further includes: determining the type of operation action indication based on at least one of the following: historical operation actions of input device 1011; the state of robot system 10; and historical operation duration of input device 1011.

[0124] It should be noted that the operation mode switching method of the robot system provided in this application embodiment belongs to the same inventive concept as the robot system described above. Its specific implementation and beneficial effects can be found in the description in the above embodiment, and will not be repeated here.

[0125] This application also provides a non-transitory computer-readable storage medium including computer instructions, such as a memory including instructions that can be executed by a processor of a device to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0126] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A robot system, characterized in that, The robot system includes: An output device is used to output instruction information, the instruction information including at least one operation action instruction, the at least one operation action instruction corresponding one-to-one with at least one operation action on the input device, and the at least one operation action corresponding one-to-one with at least one function of the robot system; The input device can be configured to follow mode or controlled mode. When configured to follow mode, it is coupled to the manipulator component so that the manipulator component follows the movement of the input device. When configured to controlled mode, it is decoupled from the manipulator component and can execute any operation action indicated by an operation action instruction. A control device is configured to, when the input device is configured in the controlled mode, detect the current operation action performed on the input device, determine the function corresponding to the current operation action, and control the robot system to activate the function corresponding to the current operation action.

2. The robot system according to claim 1, characterized in that, The types of indication information include at least one of the following: audio indication information, visual indication information, vibration indication information, and motion indication information.

3. The robot system according to claim 2, characterized in that, The visual indication information includes at least one of the following: graphic indication information, menu indication information, character indication information, color indication information, and animation indication information.

4. The robot system according to claim 3, characterized in that, The graphic indication information is arrow indication information, and the at least one operation action indication includes multiple arrows pointing in different directions. The at least one operation action includes an operation action of operating the input device along at least one operation direction, wherein the at least one operation direction corresponds one-to-one with the direction of the at least one arrow.

5. The robot system according to claim 1, characterized in that, The indication information is used to indicate at least one of the following: Operate the input device in a preset direction; The input device is operated for a preset duration; The input device is operated using a preset number of operations; Operate on a predetermined number of the input devices among the plurality of input devices; The multiple input devices are operated in a preset order; Operate on a specified input device among the plurality of input devices.

6. The robot system according to claim 1, characterized in that, The function includes at least one of the following: The function of selecting the operating mode of the robot system; The function of adjusting the image displayed by the robot system; The function of adjusting the parameters of the robot system; The function of adjusting the state of components on the robot system.

7. The robot system according to claim 1, characterized in that, The control device is also configured to configure the input device into the controlled mode in response to a controlled mode activation command.

8. The robot system according to claim 7, characterized in that, The robot system also includes an activation device, through which the controlled mode activation command is triggered.

9. The robot system according to claim 8, characterized in that, The activation device includes at least one of the following: foot pedal, button, joystick, touch screen, knob, slider.

10. The robot system according to claim 1, characterized in that, The control device is also configured to provide first tactile feedback to the input device to restrict the input device from performing any movement other than the at least one operational action.

11. The robot system according to claim 1, characterized in that, The control device is used to determine the function corresponding to the current operation when it is determined that the current operation performed by the input device has reached a travel threshold.

12. The robot system according to claim 11, characterized in that, The control device is also configured to provide a second haptic feedback to the input device to indicate how close the current operation performed by the input device is to the travel threshold.

13. The robot system according to claim 12, characterized in that, The intensity of the second tactile feedback is positively correlated with the degree of proximity.

14. The robot system according to claim 1, characterized in that, The robot system includes a plurality of the input devices, and the control device is further configured to configure at least one of the input devices into the controlled mode.

15. The robot system according to claim 1, characterized in that, The robot system includes a plurality of input devices, the plurality of input devices including a first input device configured in the follow mode and a second input device configured in the controlled mode; wherein, when the second input device is operated, a manipulator component coupled to the first input device is in a locked state.

16. The robot system according to claim 1, characterized in that, The output device is also used to output a progress indicator, which is used to indicate the progress of the robot system in executing the function corresponding to the current operation.

17. The robot system according to claim 1, characterized in that, The type of operation action indication is determined based on at least one of the following: Historical operation actions of the input device; The state of the robot system; The historical operation duration of the input device.

18. The robot system according to claim 1, characterized in that, The control device is used to determine whether the current operation action matches the operation action indicated by the at least one operation action indicator, and if so, to determine the function corresponding to the current operation action.

19. The robot system according to claim 18, characterized in that, The at least one operation action indication is an operation indication in a two-dimensional plane, and the input device can perform operation actions in a three-dimensional space. The correspondence between the operation indication in the two-dimensional plane and the operation actions in the three-dimensional space is pre-established.

20. A control method for a robot system, characterized in that, The robot system includes an input device and an output device. The output device outputs instruction information, including at least one operation action instruction. Each operation action instruction corresponds one-to-one with at least one operation action performed on the input device. Each operation action corresponds one-to-one with at least one function of the robot system. The input device can be configured in either a follower mode or a controlled mode. In follower mode, it is coupled to a manipulator component to allow the manipulator component to follow the input device's movement. In controlled mode, it is decoupled from the manipulator component and can execute any operation action instruction. The method includes: When the input device is configured in the controlled mode, the current operation performed on the input device is detected; Determine the function corresponding to the current operation action, and control the robot system to activate the function corresponding to the current operation action.

21. The method according to claim 20, characterized in that, The types of indication information include at least one of the following: audio indication information, visual indication information, vibration indication information, and motion indication information.

22. The method according to claim 21, characterized in that, The visual indication information includes at least one of the following: graphic indication information, menu indication information, character indication information, color indication information, and animation indication information.

23. The method according to claim 22, characterized in that, The graphic indication information is arrow indication information, and the at least one operation action indication includes multiple arrows pointing in different directions. The at least one operation action includes an operation action of operating the input device along at least one operation direction, wherein the at least one operation direction corresponds one-to-one with the direction of the at least one arrow.

24. The method according to claim 20, characterized in that, The indication information is used to indicate at least one of the following: Operate the input device in a preset direction; The input device is operated for a preset duration; The input device is operated using a preset number of operations; Operate on a predetermined number of the input devices among the plurality of input devices; The multiple input devices are operated in a preset order; Operate on a specified input device among the plurality of input devices.

25. The method according to claim 20, characterized in that, The function includes at least one of the following: The function of selecting the operating mode of the robot system; The function of adjusting the image displayed by the robot system; The function of adjusting the parameters of the robot system; The function of adjusting the state of components on the robot system.

26. The method according to claim 20, characterized in that, The method further includes: In response to the controlled mode activation command, the input device is configured to the controlled mode.

27. The method according to claim 26, characterized in that, The robot system also includes an activation device, through which the controlled mode activation command is triggered.

28. The method according to claim 27, characterized in that, The activation device includes at least one of the following: foot pedal, button, joystick, touch screen, knob, slider.

29. The method according to claim 20, characterized in that, The method further includes: Provide first haptic feedback to the input device to limit the input device from moving other than performing the at least one operation.

30. The method according to claim 20, characterized in that, The function for determining the current operation includes: If it is determined that the current operation performed by the input device has reached a travel threshold, the function corresponding to the current operation is determined.

31. The method according to claim 30, characterized in that, The method further includes: Provide a second haptic feedback to the input device to indicate how close the current operation performed by the input device is to the travel threshold.

32. The method according to claim 31, characterized in that, The intensity of the second tactile feedback is positively correlated with the degree of proximity.

33. The method according to claim 20, characterized in that, The robot system includes a plurality of the aforementioned input devices, and the method further includes: Configure at least one of the input devices to the controlled mode.

34. The method according to claim 20, characterized in that, The robot system includes a plurality of input devices, the plurality of input devices including a first input device configured for the follow mode and a second input device configured for the controlled mode; the method further includes: When the second input device is operated, the control actuator component coupled to the first input device is locked.

35. The method according to claim 20, characterized in that, The method further includes: The type of the operation action instruction is determined based on at least one of the following: Historical operation actions of the input device; The state of the robot system; The historical operation duration of the input device.

36. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the computer instructions are executed by the processor, they implement the steps of the method according to any one of claims 20 to 35.

Images