Robot control method and device, robot and computer readable storage medium
By recognizing robot actions and determining conflicts through a safety island processor, and controlling the robot to execute safety policies, the problem of poor robot safety control effectiveness is solved, and safety control and network protection of robots in emergency situations are realized.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- UBTECH ROBOTICS CORP LTD
- Filing Date
- 2025-12-18
- Publication Date
- 2026-04-14
AI Technical Summary
Existing robot safety control measures are ineffective, which can easily lead to accidents in various application scenarios.
The processor in the safety island collects video information, identifies the robot's current and predicted actions, determines whether there are any conflicting actions, and controls the robot to stop its current action or execute a safety policy when a conflict exists. The encryption module of the safety island protects the robot from network attacks and ensures the robot's security.
It improves the safety of robots, reduces the probability of harming targets in the surrounding environment, and ensures safe control of robots in emergency situations.
Smart Images
Figure CN121848375A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of robotics, and in particular to a robot control method, apparatus, robot, and computer-readable storage medium. Background Technology
[0002] With the rapid development of robotics technology, more and more robots are being used in various scenarios. For example, robots are used in factories for industrial production and in homes to handle household chores. However, the safety control of existing robots is relatively poor, which can easily lead to accidents in various application scenarios. Summary of the Invention
[0003] To address the aforementioned technical problems, this application provides a robot control method, apparatus, robot, and computer-readable storage medium.
[0004] In a first aspect, the present invention provides a method for controlling a robot, the method comprising: Video information is acquired through the processor in the safety island and the video processing module in the safety island. The video information includes image data of the robot and its environment. The processor in the safety island identifies the robot's current and predicted actions based on the video information; Determine whether there is a target conflict action between the current action and the predicted action; If the target conflict action exists, control the robot to stop the current action, and / or control the robot to perform the target safety action and execute the target safety policy.
[0005] In an optional implementation, identifying the robot's current action and predicted action based on the video information includes: Based on the video information, the robot's end-joint motion information and the visual relationship between the end-joint and the surrounding environment are identified; The robot's current action is determined based on the end-joint motion information; The predicted action of the robot is determined based on the visual relationship between the end joint and the surrounding environment and the robot's motion characteristic data.
[0006] In an optional implementation, determining whether there is a target conflict action between the current action and the predicted action includes: The current action and the predicted action are respectively matched with a plurality of preset conflict actions stored in the database to determine the target conflict action that matches the current action or the predicted action from the plurality of preset conflict actions.
[0007] In an optional implementation, obtaining the target security action target security policy includes: Based on the target conflict action, the target security action and target security policy are searched from multiple preset conflict actions stored in the database.
[0008] In an optional implementation, the method further includes: Determine whether the robot's motion characteristic data meets preset safety conditions; If the motion feature data does not meet the preset safety conditions, the robot is controlled to stop the current action.
[0009] In an optional implementation, the method further includes: The encryption module of the security island receives the key input by the user, pairs the key, and after successful pairing, receives a hard disk access request and reads the hard disk data of the robot according to the hard disk access request.
[0010] In an optional implementation, the method further includes: The logic unit of the safety island receives instructions from the processor and instructions from the robot control system, respectively, with the processor's instructions having a higher priority than the robot control system's instructions.
[0011] In a second aspect, the present invention provides a robot control device, the device comprising: The acquisition module is used to acquire video information through the processor of the safety island and the video processing module of the safety island. The video information includes image data of the robot and its environment. The recognition module is used to identify the robot's current action and predicted action based on the video information through the processor of the safety island; The judgment module is used to determine whether there is a target conflict action between the current action and the predicted action; The control module is configured to, if a target conflict action exists, control the robot to stop the current action, and / or control the robot to perform a target safety action and execute a target safety policy.
[0012] Thirdly, the present invention provides a robot, including a memory and a processor, wherein the memory stores a computer program, and the computer program executes the robot control method described in any of the foregoing embodiments when the processor is running.
[0013] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when run on a processor, executes the robot control method described in any of the foregoing embodiments.
[0014] The robot control method provided in this application acquires video information through a video processing module in a safety island. This video information includes image data of the robot and its environment. The processor in the safety island identifies the robot's current and predicted actions based on the video information. It then determines whether the current and predicted actions conflict with a target. If a conflict exists, the robot is controlled to stop its current action, and / or, to execute a target safety strategy. This allows the processor in the safety island to monitor in real time whether the robot's current and predicted actions affect or harm target objects in the surrounding environment, promptly stopping the current action and controlling the robot to execute corresponding target safety strategies. This reduces the probability of the robot harming target objects, and by using the safety island to constrain the robot's actions, the robot's safety is improved. Attached Figure Description
[0015] To more clearly illustrate the technical solutions of this application, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of this application and should not be considered as a limitation on the scope of protection of this application. In the various drawings, similar components are numbered similarly.
[0016] Figure 1 A flowchart illustrating the robot control method provided in this application is shown. Figure 2 An application scenario diagram of the robot provided in this application is shown; Figure 3 A structural schematic diagram of the robot provided in this application is shown; Figure 4 Another flowchart illustrating the robot control method provided in this application is shown; Figure 5 This paper illustrates another flowchart of the robot control method provided in this application; Figure 6 A schematic diagram of the control device for the robot provided in this application is shown.
[0017] Icons: 100-Robot, 200-Target Object, 300-Safety Island, 310-Processor, 320-Encryption Module, 330-Video Processing Module, 340-Camera, 350-Logic Unit, 400-Robot Control System, 500-Actuator, 600-Robot Control Device, 610-Acquisition Module, 620-Identification Module, 630-Judgment Module, 640-Control Module. Detailed Implementation
[0018] The technical solutions in this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.
[0019] The components of this application, typically described and illustrated in the accompanying drawings, can be arranged and designed in a variety of different configurations. Therefore, the following detailed description of embodiments of this application provided in the drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.
[0020] In the following, the terms “comprising,” “having,” and their cognates, which may be used in various embodiments of this application, are intended only to indicate a particular feature, number, step, operation, element, component, or combination thereof, and should not be construed as excluding, firstly, the presence of one or more other features, numbers, steps, operations, elements, components, or combinations thereof, or adding the possibility of one or more features, numbers, steps, operations, elements, components, or combinations thereof.
[0021] Furthermore, the terms "first," "second," and "third" are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.
[0022] Unless otherwise specified, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of this application pertain. Terms (such as those defined in commonly used dictionaries) shall be interpreted as having the same meaning as in their contextual meaning in the relevant technical field and shall not be construed as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of this application.
[0023] Example 1 This application provides a method for controlling a robot.
[0024] See Figure 1 The robot control methods include: Step S110: Obtain video information through the video processing module of the safety island. The video information includes image data of the robot and its environment.
[0025] See Figure 2 Robot 100 can be applied to industrial production and leisure scenarios. In various scenarios, there may be target objects 200 in the environment surrounding Robot 100. These target objects can be people, animals, objects, etc., without limitation. The movements of Robot 100's arms, legs, and body are key monitoring areas. Robot 100's arms can grasp objects, such as boxes, fruit baskets, or sharp tools. Sharp tools can be knives, guns, bows and arrows, and other dangerous tools.
[0026] See Figure 3 The safety island 300 includes a processor 310, an encryption module 320, a video processing module 330, a camera 340, and a logic unit 350. The processor 310 is connected to the encryption module 320, the logic unit 350, and the video processing module 330. The video processing module 330 is also connected to the camera 340. The logic unit 350 is connected to the robot control system 400 and the actuator 500, respectively. The processor in the safety island can be a physically isolated processor or a core of a multi-core main CPU isolated by software. This processor 310 is used to ensure the robot's safety architecture, preventing it from being affected by other cores or CPUs, ensuring that safety control strategies can be implemented in emergency situations, ensuring the robot's safety, and preventing harm to surrounding objects. In this embodiment, the robot may include two processors: one processor is used for processing the robot's main tasks, which may include motion control, visual image processing, navigation, and operation tasks; the other processor is used to monitor the robot's movements and execute safety tasks.
[0027] As an example, camera 340 captures initial video data, which is the initial video data captured in the robot's current environment, and sends the initial video data to video processing module 330. Video processing module 330 can perform preprocessing such as noise reduction and brightness adjustment on the initial video data, and then extract the video information and send the video information to processor 310.
[0028] Step S120: The processor of the safety island identifies the robot's current action and predicted action based on the video information.
[0029] In this embodiment, the current movement of the robot's arm joints or other end joints can be identified based on video information. Furthermore, based on the current movement and motion parameters of the arm joints or other end joints, the predicted future movements of the robot's arm joints or other end joints can be estimated, without limitation. The processor in the safety island predicts the robot's movement information and takes corresponding safety measures to prevent the robot from engaging in actions that could harm people or property.
[0030] See Figure 4 Step S120 includes: Step S121: Based on the video information, identify the motion information of the robot's end joints and the visual relationship between the end joints and the surrounding environment; Step S122: Determine the current action of the robot based on the end-joint motion information; Step S123: Determine the predicted action of the robot based on the visual relationship between the end joint and the surrounding environment and the motion characteristic data of the robot.
[0031] As an example, the visual relationship between the end-effector and the surrounding environment can include the relative positional relationship between the end-effector and the target object, and the robot's motion characteristic data includes robot joint torque data, robot joint velocity, robot walking speed, and the object being grasped by the robot. The relative positional relationship between the end-effector and the target object can be the distance between the end-effector and a person or object in the surrounding environment.
[0032] Step S130: Determine whether there is a target conflict action between the current action and the predicted action.
[0033] In this embodiment, the processor in the safety island determines whether there is a target conflict action between the current action and the predicted action. If there is, the process proceeds to step S140; if there is no target conflict action, the robot continues the current task.
[0034] In one embodiment, step S130 includes: The current action and the predicted action are respectively matched with a plurality of preset conflict actions stored in the database to determine the target conflict action that matches the current action or the predicted action from the plurality of preset conflict actions.
[0035] In this embodiment, all actions that may cause harm to the target object during robot use can be summarized as preset conflict actions. Corresponding preventive and protective actions can be set for each preset conflict action as the robot's preset safety strategy to ensure that the robot can take timely safety measures to avoid harming the target object.
[0036] As an example, decision scenario 1: A person enters the robot's operating space while the robot is moving, posing a risk of the robot falling and causing injury to the person. In this case, the robot should stop moving. In decision scenario 1, the robot's falling action can be considered a pre-defined conflict action.
[0037] Decision Scenario 2: The robot accidentally falls while working in the home. There are people in the area where it fell. The robot should decide and instruct its arm to make a supporting motion to avoid crushing the people. In Decision Scenario 2, the robot's fall can be considered a preset conflict action, and the robot's arm supporting motion can be considered a preset safety strategy.
[0038] Decision Scenario 3: The robot's joint motors malfunction, causing the robot to run away. The robot should stop its current action to avoid harming surrounding objects or people. In Decision Scenario 3, motor overspeed can be considered a pre-defined conflict action, and stopping the current action can be considered a pre-defined safety strategy.
[0039] Decision Scenario 4: The robot's joint motors malfunction, causing excessive torque. The robot should stop its current action to prevent surrounding objects or people from being crushed. In Decision Scenario 4, motor malfunction can be considered a pre-defined conflict action, and stopping the current action can be considered a pre-defined safety strategy.
[0040] Furthermore, a database can be pre-stored containing preset conflict actions for various personal injury or property damage scenarios, such as falling, being crushed, colliding, puncturing, squeezing, and flying kicks. For each preset conflict action, corresponding safety measures can be set in this database. Safety strategies for falling or being crushed: avoid critical parts of the body; the robot extends its hand joint for support. Safety strategies for collisions: the robot stops or decelerates. Safety strategies for punctures: the robot releases the object from its arm. Safety measures for squeezing: the robot retracts its arm. Safety strategies for flying kicks: the robot stops its actuators and decelerates.
[0041] Step S140: If the target conflict action exists, control the robot to stop the current action, and / or control the robot to execute the target safety policy.
[0042] In this embodiment, if the target conflict action exists, the processor of the safety island issues an instruction to the logic unit of the safety island. The logic unit controls the actuator to make the robot stop the current action, and / or the logic unit controls the actuator to make the robot execute the target safety policy.
[0043] It's understandable that the robot could be a humanoid robot. During operation, if an abnormal situation arises, it might harm people or objects in the surrounding environment. Once the corresponding safety strategy is activated, a safety island can take over the humanoid robot's hands or arm joints, stopping its operation. The safety island acts as an independent monitoring unit to monitor the robot's safety status. If an abnormality occurs in the robot's operating space (such as a human entering the robot's work area, triggering a safety strategy), the safety island's commands have higher priority than other commands, controlling the robot to stop its current action and execute the corresponding target safety strategy, thus improving the robot's safety.
[0044] In one embodiment, obtaining the target security policy includes: The target security policy is searched from multiple preset security policies stored in the database based on the target conflict action.
[0045] In this embodiment, a target safety strategy can be adopted according to the target conflict action of the robot to avoid the robot causing damage to the surrounding target objects and improve the robot's safety.
[0046] See Figure 5 The robot's control method also includes: Step S150: Determine whether the motion characteristic data of the robot meets the preset safety conditions; Step S160: If the motion feature data does not meet the preset safety conditions, then control the robot to stop the current action.
[0047] As an example, motion characteristic data includes robot joint torque data, robot joint velocity, etc. Preset safety conditions may include: robot joint torque data being less than or equal to a preset torque threshold, and robot joint velocity being less than or equal to a preset velocity threshold. It can be understood that if the robot joint torque data is less than or equal to the preset torque threshold, it means that the robot joint torque data is within a safe range and will not cause harm to the target object; if the robot joint torque data is greater than the preset torque threshold, it means that the robot joint torque data is out of control and will cause harm to the target object.
[0048] If the robot joint speed is less than or equal to a preset speed threshold, it indicates that the current robot joint speed is within a safe range and will not cause harm to the target object. If the robot joint speed is greater than the preset speed threshold, it indicates that the robot joint speed is out of control and will cause harm to the target object. In this embodiment, if the motion feature data meets the preset safety conditions, the robot is controlled to continue performing the current task.
[0049] In one embodiment, the robot control method further includes: The encryption module of the security island receives the key input by the user, pairs the key, and after successful pairing, receives a hard disk access request and reads the hard disk data of the robot according to the hard disk access request.
[0050] In this embodiment, the encryption module of the security island protects the robot from information intrusion, thereby ensuring control over the robot. For example, the encryption module can prevent network attacks on the robot and protect it from network intrusion. The encryption module authorizes access to the robot's hardware interfaces (USB, Ethernet, etc.), preventing unauthorized access to hard drive-related data such as videos, images, and text. Once the robot's hard drive is removed, the data cannot be read or copied in any other way. As an example, the encryption protection process is as follows: When a user logs in locally or via the network, the user first accesses the encryption module. The encryption module verifies the key entered by the user. Only after successful key pairing is the user allowed to access the hard drive data on the motherboard. Thus, accessing the encrypted hard drive requires user verification, and the key cannot be obtained through a third party, thereby ensuring the security of the hard drive data.
[0051] In one embodiment, the robot control method further includes: The logic unit of the safety island receives instructions from the processor and instructions from the robot control system, respectively, with the processor's instructions having a higher priority than the robot control system's instructions.
[0052] In this embodiment, the logic unit of the safety island is used to receive instructions from the processor of the safety island and instructions from the robot control system, and output control instructions to the actuators. The processor instructions of the safety island have higher priority than the robot control system. The actuators may include various joint actuators, such as arm joint actuators, leg joint actuators, etc., which are not limited here.
[0053] To clarify, the robot control system's commands include robot walking (slow walking, fast walking, backward walking), handling (squatting, standing up), arm movements (extension, rotation), and dexterous hand movements (such as grasping, releasing). Actuators refer to the robot's joints and dexterous hands. The safety island's processor commands mainly include: stopping the current action, retracting the arm or leg, and releasing the object held in the hand. Both the safety island and the robot control system can control the actuators. Under any circumstances, the safety island's commands have higher priority than the robot control system's commands. The robot's actions can be terminated at any time using the safety island's commands, ensuring the safety of objects around the robot.
[0054] The robot control method provided in this embodiment acquires video information through the video processing module of the safety island. The video information includes image data of the robot and its environment. The processor of the safety island identifies the robot's current action and predicted action based on the video information. It determines whether the current action and the predicted action conflict with a target. If a conflict exists, the robot is controlled to stop the current action, and / or, the robot is controlled to execute a target safety strategy. In this way, the processor of the safety island monitors in real time whether the robot's current action and predicted action affect or harm target objects in the surrounding environment, stops the current action in time, and controls the robot to execute the corresponding target safety strategy, reducing the probability of the robot causing harm to the target object. The safety island is used to constrain the robot's actions and improve the robot's safety.
[0055] Example 2 In addition, this application provides a control device for a robot.
[0056] like Figure 6 As shown, the robot's control device 600 includes: The acquisition module 610 is used to acquire video information through the video processing module of the safety island, the video information including image data of the robot and its environment; The recognition module 620 is used to recognize the robot's current action and predicted action based on the video information through the processor of the safety island; The judgment module 630 is used to determine whether there is a target conflict action between the current action and the predicted action; The control module 640 is configured to, if the target conflict action exists, control the robot to stop the current action, and / or control the robot to execute the target safety strategy.
[0057] In one embodiment, the recognition module 620 is used to recognize the end joint motion information of the robot and the visual relationship between the end joint and the surrounding environment based on the video information. The robot's current action is determined based on the end-joint motion information; The predicted action of the robot is determined based on the visual relationship between the end joint and the surrounding environment and the robot's motion characteristic data.
[0058] In one embodiment, the determination module 630 is used to match the current action and the predicted action with a plurality of preset conflict actions stored in the database, so as to determine the target conflict action that matches the current action or the predicted action from the plurality of preset conflict actions.
[0059] In one embodiment, the robot's control device 600 further includes: The search module is used to search for the target security policy from multiple preset security policies stored in the database based on the target conflict action.
[0060] In one embodiment, the judgment module 630 is further configured to determine whether the motion characteristic data of the robot meets preset safety conditions; If the motion feature data does not meet the preset safety conditions, the robot is controlled to stop the current action.
[0061] In one embodiment, the robot's control device 600 further includes: The processing module is used to receive the key input by the user through the encryption module of the security island, pair the key, and after successful pairing, receive a hard disk access request and read the hard disk data of the robot according to the hard disk access request.
[0062] In one embodiment, the robot's control device 600 further includes: The receiving module is used to receive instructions from the processor and instructions from the robot control system through the logic unit of the safety island, respectively, wherein the instructions from the processor have a higher priority than the instructions from the robot control system.
[0063] The robot control device 600 provided in this embodiment can implement the robot control method provided in Embodiment 1. To avoid repetition, it will not be described again here.
[0064] The robot control device provided in this embodiment acquires video information through the video processing module of the safety island. The video information includes image data of the robot and its environment. The processor of the safety island identifies the robot's current action and predicted action based on the video information. It determines whether the current action and the predicted action conflict with a target. If a conflict exists, it controls the robot to stop the current action and / or controls the robot to execute a target safety strategy. In this way, the processor of the safety island monitors in real time whether the robot's current action and predicted action affect or harm target objects in the surrounding environment, stops the current action in time, and controls the robot to execute the corresponding target safety strategy, reducing the probability of the robot causing harm to the target object. The safety island is used to constrain the robot's actions and improve the robot's safety.
[0065] Example 3 Furthermore, this application provides a robot, including a memory and a processor, wherein the memory stores a computer program, and the computer program executes the robot control method provided in Embodiment 1 when running on the processor.
[0066] The electronic device provided in this embodiment can implement the robot control method provided in Embodiment 1. To avoid repetition, it will not be described again here.
[0067] Example 4 This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the robot control method provided in Embodiment 1.
[0068] In this embodiment, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.
[0069] The computer-readable storage medium provided in this embodiment can implement the robot control method provided in Embodiment 1. To avoid repetition, it will not be described again here.
[0070] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes that element.
[0071] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0072] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.
Claims
1. A method for controlling a robot, characterized in that, The method includes: Video information is obtained through the video processing module of the safety island, and the video information includes image data of the robot and its environment. The processor in the safety island identifies the robot's current and predicted actions based on the video information; Determine whether there is a target conflict action between the current action and the predicted action; If the target conflict action exists, control the robot to stop the current action, and / or control the robot to execute the target safety policy.
2. The method according to claim 1, characterized in that, The step of identifying the robot's current action and predicted action based on the video information includes: Based on the video information, the robot's end-joint motion information and the visual relationship between the end-joint and the surrounding environment are identified; The robot's current action is determined based on the end-joint motion information; The predicted action of the robot is determined based on the visual relationship between the end joint and the surrounding environment and the robot's motion characteristic data.
3. The method according to claim 2, characterized in that, The step of determining whether there is a target conflict action between the current action and the predicted action includes: The current action and the predicted action are respectively matched with a plurality of preset conflict actions stored in the database to determine the target conflict action that matches the current action or the predicted action from the plurality of preset conflict actions.
4. The method according to claim 3, characterized in that, Obtaining the target security policy includes: The target security policy is searched from multiple preset security policies stored in the database based on the target conflict action.
5. The method according to claim 1, characterized in that, The method further includes: Determine whether the robot's motion characteristic data meets preset safety conditions; If the motion feature data does not meet the preset safety conditions, the robot is controlled to stop the current action.
6. The method according to claim 1, characterized in that, The method further includes: The encryption module of the security island receives the key input by the user, pairs the key, and after successful pairing, receives a hard disk access request and reads the hard disk data of the robot according to the hard disk access request.
7. The method according to any one of claims 1-6, characterized in that, The method further includes: The logic unit of the safety island receives instructions from the processor and instructions from the robot control system, respectively, with the processor's instructions having a higher priority than the robot control system's instructions.
8. A control device for a robot, characterized in that, The device includes: The acquisition module is used to acquire video information through the video processing module of the safety island. The video information includes image data of the robot and its environment. The recognition module is used to identify the robot's current action and predicted action based on the video information through the processor of the safety island; The judgment module is used to determine whether there is a target conflict action between the current action and the predicted action; The control module is configured to, if a target conflict action exists, control the robot to stop the current action, and / or control the robot to execute a target safety policy.
9. A robot, characterized in that, It includes a memory and a processor, the memory storing a computer program that executes the robot control method according to any one of claims 1 to 7 when the processor is running.
10. A computer-readable storage medium, characterized in that, It stores a computer program that, when run on a processor, executes the robot control method according to any one of claims 1 to 7.