Method, apparatus and electronic device for safe navigation and operation of a robot

Abstract

The application provides a safety navigation and operation method and device of a robot and electronic equipment, and relates to the technical field of artificial intelligence. The implementation scheme is as follows: in response to a first instruction for requesting the robot to perform an operation, triggering the robot to start navigation and operation to perform the operation of the first instruction; in the case of robot movement, determining at least one first object located in a current region corresponding to a current position of the robot based on the current position; determining a maximum instantaneous acceleration of the robot movement based on the maximum damage degree of each first object and the mass of the robot; determining the maximum operation degree of the robot allowed to operate each first object based on the maximum damage degree of each first object; and controlling the robot to perform navigation and operation based on the maximum instantaneous acceleration and the maximum operation degree.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus and electronic device for safe navigation and operation of a robot. Background Technology

[0002] In recent years, with the rapid development of intelligent robots, robots have been widely used in many fields, especially in indoor environments, where autonomous robot navigation has become a research hotspot. Existing robot navigation technologies mainly include visual navigation, LiDAR navigation, and inertial navigation, among which visual navigation is widely adopted due to its low cost and powerful environmental perception capabilities. Traditional visual navigation systems typically rely on image sensors to collect environmental information and achieve navigation through feature extraction and path planning algorithms. However, this approach is easily affected by factors such as changes in lighting and obstructions in complex environments, leading to reduced navigation accuracy.

[0003] To address the aforementioned issues, language navigation technology has been introduced into the field of robotics, optimizing robot control through language commands. However, language navigation alone suffers from poor positioning accuracy and limited environmental interactivity, especially in complex indoor environments where the lack of surrounding information leads to vague and imprecise understanding of language instructions by the robot. This can result in dangerous actions that could damage surrounding objects during the robot's execution of commands. Therefore, how to control robots for safe navigation and operation is a key technical problem that needs to be solved in this field. Summary of the Invention

[0004] This invention provides a method, apparatus, and electronic device for safe navigation and operation of a robot, which can solve at least one of the above-mentioned technical problems.

[0005] According to one aspect of the present invention, a method for safe navigation and operation of a robot is provided, comprising:

[0006] In response to a first instruction requesting the robot to perform an operation, the robot is triggered to begin navigation and operation to execute the operation of the first instruction;

[0007] When the robot is moving, based on the robot's current position, at least one first object located within the current area corresponding to the current position is determined;

[0008] Based on the maximum damage force of each of the first objects and the mass of the robot, the maximum instantaneous acceleration of the robot's motion is determined;

[0009] Based on the maximum damage force of each of the first objects, the maximum operating force by which the robot is allowed to operate on each of the first objects is determined.

[0010] The robot is controlled to navigate and operate based on the maximum instantaneous acceleration and the maximum operating force.

[0011] According to another aspect of the present invention, a safe navigation and operation device for a robot is provided, comprising:

[0012] A triggering module is used to respond to a first instruction requesting the robot to perform an operation, triggering the robot to begin navigation and operation to execute the operation of the first instruction;

[0013] The first object determination module is used to determine at least one first object located in the current area corresponding to the current position, based on the current position of the robot when the robot is moving.

[0014] An acceleration determination module is used to determine the maximum instantaneous acceleration of the robot's motion based on the maximum damage force of each of the first objects and the mass of the robot.

[0015] The operation force determination module is used to determine the maximum operation force that the robot is allowed to operate on each of the first objects based on the maximum damage force of each of the first objects.

[0016] The navigation and operation module is used to control the robot to navigate and operate based on the maximum instantaneous acceleration and the maximum operating force.

[0017] By employing the technical solution of this invention, in response to a first command requesting the robot to perform an operation, the robot is triggered to begin navigation and operation to execute the operation of the first command. While the robot is moving, based on the robot's current position, at least one first object is identified within the current area corresponding to that position. Based on the maximum damage force of each first object and the robot's mass, the maximum instantaneous acceleration of the robot's movement is determined. Based on the maximum damage force of each first object, the maximum operating force by which the robot is allowed to operate on each first object is determined. Based on the maximum instantaneous acceleration and the maximum operating force, the robot is controlled to navigate and operate. In this way, damage to surrounding objects by the robot can be minimized when the robot executes commands, improving the safety performance of robot navigation and operation.

[0018] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0019] The accompanying drawings are provided for a better understanding of this solution and do not constitute a limitation of the invention. Wherein:

[0020] Figure 1This is a flowchart of a robot safety navigation and operation method according to an embodiment of the present invention;

[0021] Figure 2 This is a schematic diagram of a robot control process according to an embodiment of the present invention;

[0022] Figure 3 This is a structural block diagram of a robot safety navigation and operation device according to an embodiment of the present invention;

[0023] Figure 4 This is a block diagram of an electronic device used to implement the methods of embodiments of the present invention. Detailed Implementation

[0024] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0025] Figure 1 This is a flowchart of a robot safety navigation and operation method according to an embodiment of the present invention.

[0026] like Figure 1 As shown, the safe navigation and operation method of this robot may include:

[0027] S110, in response to the first instruction requesting the robot to perform an operation, triggers the robot to begin navigation and operation to execute the operation of the first instruction;

[0028] S120, while the robot is moving, based on the robot's current position, determine at least one first object located in the current area corresponding to the current position;

[0029] S130, based on the maximum damage force of each first object and the mass of the robot, determine the maximum instantaneous acceleration of the robot's motion;

[0030] S140, based on the maximum damage force of each first object, determine the maximum operating force that the robot is allowed to operate on each first object.

[0031] The S150 controls the robot's navigation and operation based on maximum instantaneous acceleration and maximum operating force.

[0032] For example, the first instruction can be a voice instruction or a text instruction. Alternatively, it can be a voice-text instruction obtained by performing speech recognition on the voice instruction.

[0033] For example, the first instruction may be an instruction requesting the robot to perform an operation on one or more objects, or it may be an instruction requesting the robot to perform an operation on itself.

[0034] For example, the first object may include the object in the first instruction.

[0035] For example, such as Figure 2 As shown, the operation provides voice commands to the visual language navigation device in the mobile robot control system through the control terminal. The visual language navigation device recognizes the voice commands and executes the method provided in this embodiment of the invention. Then, it provides the obtained information, such as the robot's maximum instantaneous acceleration and maximum operating force, as well as the attribute information of surrounding objects, to the computing terminal. The computing terminal provides navigation point information to the mobile robot control system. The mobile robot control system controls the mobile robot body to move and operate through movement commands.

[0036] For example, the current area can be a circular area centered on the robot and within a set radius. This radius can be the robot's working radius. There can be one or more objects located within this circular area.

[0037] For example, the maximum damaging force can be considered as the critical force at which an object will be damaged. Applying this force or a force greater than this force to an object will cause the object to be damaged; applying a force less than this force to an object will not cause the object to be damaged.

[0038] For example, a minimum value is determined from the maximum damage forces of each first object, and the ratio between the minimum value and the mass of the robot is used as the maximum instantaneous acceleration of the robot's motion.

[0039] For example, the average value is obtained by averaging the maximum damage force of each first object, and the ratio between this average value and the mass of the robot is used as the maximum instantaneous acceleration of the robot's motion.

[0040] In this way, even if the robot collides with these objects during its movement, the objects will not be damaged.

[0041] For example, if the first instruction is to request the robot to operate on the second object, then the maximum operating force that the robot is allowed to operate on each of the first objects and the second object is determined based on the maximum damage force of each of the first objects and the maximum damage force of the second object.

[0042] For example, the minimum of the maximum damage force to each of the first and second objects can be used as the robot's maximum operating force. Alternatively, since the robot operates on the second object, the maximum damage force to the second object can be used as the robot's maximum operating force.

[0043] For example, based on the first instruction, a navigation route planned by the robot to execute the first instruction is determined. Based on the navigation route, a target object that the robot will come into contact with is determined among the various first objects. The minimum value between the maximum damage force of the target object and the maximum damage force of the second object is used as the maximum operating force of the robot.

[0044] Understandably, the second object is different from each of the first objects, or the second object is one of the first objects.

[0045] For example, the robot is controlled to move according to the navigation path based on the maximum instantaneous acceleration.

[0046] Following the previous example, when the robot reaches the target position and begins to operate on the second object, the robot operates on the second object based on the maximum operating force.

[0047] For example, if the first instruction requests the robot to combine a second object with a third object, the maximum operating force of the robot on the second object is determined based on the maximum damage force of the second object and the maximum safe combination force of the second and third objects. This operation can be a pushing force, a throwing force, a clamping force, or two of these three forces. For instance, if the robot first clamps the second object and then pushes, pulls, or throws it to combine it with the third object, the maximum damage force of the second object is used as the robot's clamping force, and the maximum safe combination force of the second and third objects is used as the pushing or throwing force.

[0048] For example, combining a second object with a third object could be: pushing a chair onto a table, placing a bowl on the tabletop, and putting a basketball into a basket, etc.

[0049] For example, the maximum safe combination force of the second object and the third object can be understood as follows: when the second object and the third object are combined with the maximum safe combination force and a force less than that force, the second object and the third object will not be damaged; when the second object and the third object are combined with a force greater than the maximum safe combination force, the second object and the third object will be damaged.

[0050] According to the above implementation method, in response to a first instruction requesting the robot to perform an operation, the robot is triggered to begin navigation and operation to execute the operation of the first instruction. While the robot is moving, based on the robot's current position, at least one first object located within the current area corresponding to the current position is identified. Based on the maximum damage force of each first object and the robot's mass, the maximum instantaneous acceleration of the robot's movement is determined. Based on the maximum damage force of each first object, the maximum operating force by which the robot is allowed to operate on each first object is determined. Based on the maximum instantaneous acceleration and the maximum operating force, the robot is controlled to navigate and operate. In this way, damage to surrounding objects by the robot can be avoided as much as possible when the robot executes instructions, improving the safety performance of robot navigation and operation.

[0051] In one embodiment, the step of triggering the robot to start navigation and operation includes: if the first instruction is a request for the robot to operate on the second object, determining a third object involved in the robot's operation on the second object based on the first instruction; determining the execution risk of the first instruction based on the attribute information of the second object, the attribute information of the third object, and the safety interaction information between the second object and the third object; and triggering the robot to start navigation and operation if the execution risk meets the preset safety execution conditions.

[0052] For example, the first instruction is to request the robot to operate on the second object, including requesting the robot to operate only on the second object, and the robot to combine the second object with the third object.

[0053] For example, the third object can be an object that combines the second object, or it can be an object that the robot needs to operate on before it can operate on the second object.

[0054] For example, the attribute information of the second object may include the object hardware attributes and the object vulnerability attributes of the second object.

[0055] For example, the attribute information of the third object may include the object hardware attributes and the object vulnerability attributes of the third object.

[0056] The object's hardware attributes include its category, material, location, and color. The object's vulnerability attributes include information on whether the object is vulnerable, its maximum damage force, and the robot's movement speed when the object is damaged by the robot.

[0057] For example, the secure interaction information between the second object and the third object includes whether the second object and the third object can be combined, and if they can be combined, the maximum secure combination strength of the second object and the third object.

[0058] For example, a pre-trained model can be used to process these three pieces of information and the instruction to obtain the execution risk of the first instruction.

[0059] For example, the vulnerability attributes of the second object, the vulnerability attributes of the third object, and the maximum safe combined strength of the second and third objects can be used for mapping processing respectively. Then, the three mapping results are weighted and summed to obtain the execution risk of the first instruction.

[0060] For example, a safe execution condition could be that the execution risk is less than a preset risk threshold. If the execution risk is less than the preset risk threshold, the robot is triggered to begin navigation and operation. If the execution risk is greater than the preset risk threshold, a warning indication to refuse execution is issued. This warning indication may include the aforementioned execution risk.

[0061] In this example, by combining information such as the object's attribute labels, fragility labels, and safe interaction force, the risk of the instruction is assessed. If the instruction may cause damage to the object, the robot issues a warning and immediately executes the task corresponding to the instruction. This improves the safety performance of the robot's navigation and operation.

[0062] In one implementation, determining the execution risk of the first instruction based on the attribute information of the second object, the attribute information of the third object, and the security interaction information between the second and third objects includes: inputting the first instruction, the attribute information of the second object, the attribute information of the third object, and the security interaction information between the second and third objects into a secure language operation model to obtain the execution risk of the first instruction output by the secure language operation model.

[0063] The training process of the secure language operation model includes: inputting the first training sample into the secure language operation model to obtain the first risk assessment result output by the secure language operation model; inputting the first training sample into the pre-trained large language model to obtain the second risk assessment result output by the large language model; determining the loss function based on the first risk assessment result and the second risk assessment result; and adjusting the model parameters of the secure language operation model based on the loss function.

[0064] The first training sample includes an instruction sample, attribute information of a first object sample, attribute information of a second object sample, and safety interaction information between the first and second object samples. The instruction sample is an instruction requesting the robot to operate on the first object sample, and the second object sample is an object involved in the operation of the first object sample by the robot, determined based on the instruction sample.

[0065] For example, the first instruction, the attribute information of the second object, the attribute information of the third object, and the safe interaction information between the second and third objects are input into the secure language manipulation model. The secure language manipulation model uses a feature extraction module (such as CNN or Transformer) to extract features from the object's type, material, and location, obtaining corresponding feature representations. Simultaneously, the secure language manipulation model uses a natural language processing (NLP) module to parse the first instruction, obtaining the key verbs and object descriptions within the instruction. Then, the secure language manipulation model uses the extracted features and parsed information to predict the execution risk of the first instruction.

[0066] For example, a score can be used to represent execution risk; the higher the execution risk, the higher the score.

[0067] Understandably, a Large Language Model (LLM) is used to infer the potential impact of instructions to determine whether operating on the instruction sample will result in damage to the object. That is, the second risk assessment result is used as ground truth or labeled data during model training. The first risk assessment result is used as predicted data during model training. In this way, the training data used during training does not need to be manually labeled to obtain ground truth data, and using a large language model to obtain ground truth data improves labeling efficiency, thereby increasing training speed.

[0068] According to the above implementation method, a safe language operation model can be trained accurately and quickly, and the safe language operation model can be used to process the first instruction, the attribute information of the second object, the attribute information of the third object, and the safe interaction information between the second object and the third object to obtain the execution risk of the first instruction.

[0069] In one implementation, the attribute information includes object hardware attributes and object vulnerability attributes; the object hardware attributes include object category, object material, object position, and object color; the object vulnerability attributes include information on whether the object is vulnerable, the maximum damage force of the object, and the robot's movement speed when the object is damaged by the robot; the safety interaction information includes whether the corresponding two objects can be combined, and the maximum safe combination force between the two objects.

[0070] For example, the object hardware properties of any fourth object can be determined in the following way:

[0071] The RAM and YOLO models are used to process the RGB image including the fourth object to obtain the first and second classification results of the fourth object. A material estimation model is used to process the RGB image including the fourth object to obtain the object material. Back projection calculation is performed on the RGB image including the fourth object and the corresponding depth image to obtain the object position of the fourth object. Color extraction is performed on the RGB image including the fourth object to obtain the object color of the fourth object. The first and second classification results of the fourth object, the object material, the object position, and the object color are input into the multimodal classification model to obtain the object category of the fourth object output by the multimodal classification model.

[0072] For example, a multimodal classification model includes a multimodal feature extraction module, a feature fusion module, and a classification module.

[0073] For example, the multimodal feature extraction module employs different sub-networks to extract specific features of the object for each feature. For instance, it uses a convolutional neural network (CNN) to extract RGB image features, a PointNet or Transformer network to extract point cloud features, and fully connected layers to process label confidence scores and material features.

[0074] For example, the feature fusion module integrates features from different modalities into a unified feature representation through concatenation, attention mechanisms, or other fusion strategies.

[0075] For example, the classification module: after fusing features, outputs a score for each object label through a fully connected network to select the optimal label.

[0076] Based on the example above, a multimodal classification model is used, taking the two classification results of the object, the object material, the object spatial pose matrix, the object volume, and the object color as inputs, scoring the label of each object, and selecting the label with the highest score as the final label of the object.

[0077] For example, the object category of each object can be calculated according to preset steps, thereby constructing a sliding window sequence, saving the object category label at each moment, and selecting the label that appears most frequently as the current final object category, so as to ensure the accuracy of the object category.

[0078] According to the above implementation method, the object hardware attributes can be accurately calculated.

[0079] In one embodiment, the method further includes:

[0080] The identification information of each fifth object in the robot's allowed movement area is used to identify each node, and the safety interaction information between the two objects corresponding to any two nodes is used as the edge between the two nodes to construct an object safety network graph.

[0081] Based on the identification information of the second object and the third object, search for the security interaction information between the second object and the third object in the object security network graph.

[0082] In one embodiment, the method further includes: inputting the hardware attributes of the fourth object and a first prompt question inquiring about whether the fourth object is vulnerable into a large language model to obtain information on whether the second object is vulnerable, output by the large language model; inputting the hardware attributes of the fourth object and a second prompt question inquiring about the damage intensity of the fourth object into the large language model to obtain the maximum damage intensity of the second object, output by the large language model; inputting the hardware attributes of the fourth object, the mass of the robot, and a third prompt question inquiring about the robot's movement speed when the fourth object is damaged by the robot into the large language model to obtain the robot's movement speed when the fourth object is damaged by the robot, output by the large language model; wherein the fourth object includes any one of the first object, the second object, the third object, a sample of the first object, and a sample of the second object.

[0083] For example, the fourth object may be an object within the robot's permitted movement area.

[0084] For example, each object in the robot's allowed movement area is used as a node, and the attribute information of each object is used as node information. The safety interaction information between the two objects corresponding to any two nodes is used as the edge between the two nodes, thus constructing an object safety network graph.

[0085] For example, attribute information of each object can be obtained from the object security network diagram.

[0086] For example, after obtaining the attribute information of each object, a large language model (LLM) is used to perform a hazard analysis on each object. Specifically, the LLM is used to ask the following questions:

[0087] The first prompt question is, for example, "Is this object made of a fragile material?"

[0088] The second prompt question is, for example: "What force (unit: Newton) would cause an object to be damaged (defined as the safe interaction force or the maximum damage force)?"

[0089] The third prompt question is, for example: "The robot has a mass of X kg. In the current scenario, at what speed will the robot damage the object?"

[0090] Therefore, based on the above answers, three vulnerability labels are generated for each object: whether it is vulnerable, maximum damage force (maximum safe interaction force), and maximum safe speed.

[0091] According to the above implementation method, the vulnerability attributes of each object are generated using a large language model.

[0092] In one embodiment, the method further includes: determining a corresponding circular region centered on the position information of the fifth object and with a radius determined by the size of the machine, and determining each sixth object within the circular region; for each sixth object, inputting the attribute information of the fifth object and the attribute information of the sixth object, as well as a fourth prompt question inquiring whether the fifth and sixth objects can be combined, into a large language model to obtain information output by the large language model indicating whether the fifth and sixth objects can be combined; if the information indicating that the fifth and sixth objects can be combined is obtained, inputting a fifth prompt question inquiring about the maximum safe combination strength when the fifth and sixth objects are combined into the large language model to obtain the maximum safe combination strength when the fifth and sixth objects are combined, as output by the large language model; and determining safe interaction information between the fifth and sixth objects based on the information indicating whether the fifth and sixth objects can be combined, and the maximum safe combination strength when the fifth and sixth objects are combined, if they can be combined.

[0093] For example, each object is represented as a node in the object safety network graph, and the node information includes the object's location, three vulnerable tags, material, color and other attributes.

[0094] For example, with each object as the center (referred to as object A), objects within a circle (referred to as object B) are defined around it with a radius of twice the maximum width of the robot's body.

[0095] For example, for objects A and B, the following queries are made using LLM:

[0096] The fourth prompt question is, for example, "Can these two objects be combined or interacted with?" (e.g., pushing a chair next to a table, placing a bowl on the table, or putting a basketball into a hoop).

[0097] If combinations are possible, ask the fifth prompt question, such as: "What is the maximum safe combined force when combined?" (unit: Newtons).

[0098] Through the above steps, a safe interaction edge connecting object A and object B is generated, which contains two key pieces of information: whether they can be combined, and the maximum safe combination force (the force that cannot be exceeded when combining objects, in Newtons).

[0099] Based on the above implementation method, edges between nodes in the object security network graph can be constructed.

[0100] In one implementation, determining the maximum instantaneous acceleration of the robot's motion based on the maximum damage force of each first object and the robot's mass may include: determining the maximum instantaneous acceleration of the robot's motion based on the ratio between the average of the maximum damage forces of each first object and the robot's mass.

[0101] According to the above implementation method, the robot moves at the maximum instantaneous acceleration, and even if it collides with surrounding objects, it will not damage the surrounding objects.

[0102] Figure 3 This is a structural block diagram of a robot safety navigation and operation device according to an embodiment of the present invention.

[0103] like Figure 3 As shown, the robot's safety navigation and operation device may include:

[0104] Trigger module 310 is used to respond to a first instruction requesting the robot to perform an operation, triggering the robot to start navigation and operation to execute the operation of the first instruction;

[0105] The first object determination module 320 is used to determine at least one first object located in the current area corresponding to the current position, based on the current position of the robot when the robot is moving.

[0106] The acceleration determination module 330 is used to determine the maximum instantaneous acceleration of the robot's motion based on the maximum damage force of each of the first objects and the mass of the robot.

[0107] The operation force determination module 340 is used to determine the maximum operation force by which the robot is allowed to operate on each of the first objects based on the maximum damage force of each of the first objects.

[0108] The navigation and operation module 350 is used to control the robot to navigate and operate based on the maximum instantaneous acceleration and the maximum operating force.

[0109] In one embodiment, the triggering module 310 includes:

[0110] The second object determination unit is used to determine, based on the first instruction, a third object involved in the process of the robot operating the second object when the first instruction is a request for the robot to operate on the second object.

[0111] The execution risk determination unit is used to determine the execution risk of the first instruction based on the attribute information of the second object, the attribute information of the third object, and the security interaction information between the second object and the third object.

[0112] The triggering unit is used to trigger the robot to start navigation and operation when the execution risk meets the preset safe execution conditions.

[0113] In one implementation, the execution risk determination unit is specifically used for:

[0114] The first instruction, the attribute information of the second object, the attribute information of the third object, and the security interaction information between the second object and the third object are input into the security language operation model to obtain the execution risk of the first instruction output by the security language operation model.

[0115] The training process of the secure language operation model includes:

[0116] The first training sample is input into the secure language operation model to obtain the first risk assessment result output by the secure language operation model;

[0117] The first training sample is input into the pre-trained large language model to obtain the second risk assessment result output by the large language model.

[0118] Based on the first risk assessment result and the second risk assessment result, determine the loss function;

[0119] Based on the loss function, adjust the model parameters of the secure language operation model;

[0120] The first training sample includes an instruction sample, attribute information of a first object sample, attribute information of a second object sample, and safety interaction information between the first object sample and the second object sample. The instruction sample is an instruction requesting the robot to operate on the first object sample, and the second object sample is an object determined based on the instruction sample during the operation of the robot on the first object sample.

[0121] In one implementation, the attribute information includes object hardware attributes and object vulnerability attributes;

[0122] The object's hardware attributes include object category, object material, object position, and object color;

[0123] The object's vulnerability attributes include information on whether the object is vulnerable, the maximum damage force of the object, and the robot's movement speed when the object is damaged by the robot.

[0124] The safety interaction information includes whether the two corresponding objects can be combined, and the maximum safe combination force between the two objects.

[0125] In one embodiment, the above-described apparatus further includes:

[0126] The first vulnerability information determination module is used to input the hardware attributes of the fourth object and a first prompt question asking whether the fourth object is vulnerable into the large language model, and obtain the vulnerability information of the second object output by the large language model.

[0127] The second vulnerability information determination module is used to input the hardware attributes of the fourth object and a second prompt question that queries the damage level of the fourth object into the large language model to obtain the maximum damage level of the second object output by the large language model.

[0128] The third vulnerability information determination module is used to input the object hardware attributes of the fourth object, the mass of the robot, and a third prompt question that queries the speed of the robot's movement when the fourth object is damaged by the robot into the large language model, so as to obtain the speed of the robot's movement when the fourth object is damaged by the robot output by the large language model.

[0129] The fourth object includes any one of the first object, the second object, the third object, the first object sample, and the second object sample.

[0130] In one embodiment, the above-mentioned device further includes:

[0131] The network graph construction module is used to identify each node by using the identification information of each fifth object in the robot's allowed motion area, and to construct an object safety network graph by using the safety interaction information between the two objects corresponding to any two nodes as the edge between the two nodes.

[0132] The edge information lookup module is used to look up security interaction information between the second object and the third object in the object security network graph based on the identification information of the second object and the identification information of the third object.

[0133] In one embodiment, the above-mentioned device further includes:

[0134] The region determination module is used to determine a corresponding circular region with the position information of the fifth object as the center and the radius determined by the size of the machine, and to determine each sixth object in the circular region.

[0135] The first combination information determination module is used to input the attribute information of the fifth object and the attribute information of the sixth object, as well as the fourth prompt question asking whether the fifth object and the sixth object can be combined, into the large language model for each of the sixth objects, so as to obtain the information output by the large language model on whether the fifth object and the sixth object can be combined.

[0136] The second combination information determination module is used to input a fifth prompt question that asks about the maximum safe combination force when the fifth object and the sixth object are combined into the large language model when the information that the fifth object and the sixth object can be combined is obtained, so as to obtain the maximum safe combination force when the fifth object and the sixth object are combined as output by the large language model.

[0137] The interaction information determination module is used to determine the safe interaction information between the fifth object and the sixth object based on information about whether the fifth object and the sixth object can be combined, and the maximum safe combination force when the fifth object and the sixth object are combined if they can be combined.

[0138] In one embodiment, the acceleration determination module 330 is specifically used to: determine the maximum instantaneous acceleration of the robot's motion based on the ratio between the average of the maximum damage force of each of the first objects and the mass of the robot.

[0139] The specific functions and examples of each module and submodule of the system in this embodiment of the invention can be found in the relevant descriptions of the corresponding steps in the above method embodiments, and will not be repeated here.

[0140] The acquisition, storage, and application of user personal information involved in the technical solution of this invention all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0141] According to embodiments of the present invention, the present invention also provides a system and a readable storage medium.

[0142] Figure 4 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0143] like Figure 4 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0144] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0145] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as a robot safety navigation and manipulation method. For example, in some embodiments, the robot safety navigation and manipulation method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the robot safety navigation and manipulation method described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the robot safety navigation and manipulation method by any other suitable means (e.g., by means of firmware).

[0146] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0147] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0148] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0149] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0150] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0151] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0152] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0153] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for safe navigation and operation of a robot, characterized in that, include: In response to a first instruction requesting the robot to perform an operation, the robot is triggered to begin navigation and operation to execute the operation of the first instruction; When the robot is moving, based on the robot's current position, at least one first object located within the current area corresponding to the current position is determined; Based on the maximum damage force of each of the first objects and the mass of the robot, the maximum instantaneous acceleration of the robot's motion is determined. The maximum damage force refers to the critical force at which the object is damaged. If a force greater than or equal to the maximum damage force is applied to the object, the object will be damaged. Based on the maximum damage force of each of the first objects, the maximum operating force by which the robot is allowed to operate on each of the first objects is determined. The robot is controlled to navigate and operate based on the maximum instantaneous acceleration and the maximum operating force. The triggering of the robot to begin navigation and operation includes: When the first instruction is to request the robot to operate on the second object, a third object is determined based on the first instruction during the robot's operation on the second object; Based on the attribute information of the second object, the attribute information of the third object, and the security interaction information between the second object and the third object, the execution risk of the first instruction is determined. If the execution risk meets the preset safe execution conditions, the robot is triggered to begin navigation and operation; The step of determining the execution risk of the first instruction based on the attribute information of the second object, the attribute information of the third object, and the security interaction information between the second object and the third object includes: The first instruction, the attribute information of the second object, the attribute information of the third object, and the security interaction information between the second object and the third object are input into the security language operation model to obtain the execution risk of the first instruction output by the security language operation model. The training process of the secure language operation model includes: The first training sample is input into the secure language operation model to obtain the first risk assessment result output by the secure language operation model; The first training sample is input into the pre-trained large language model to obtain the second risk assessment result output by the large language model. Based on the first risk assessment result and the second risk assessment result, determine the loss function; Based on the loss function, adjust the model parameters of the secure language operation model; The first training sample includes an instruction sample, attribute information of a first object sample, attribute information of a second object sample, and safety interaction information between the first object sample and the second object sample. The instruction sample is an instruction requesting the robot to operate on the first object sample, and the second object sample is an object determined based on the instruction sample during the operation of the robot on the first object sample.

2. The method according to claim 1, characterized in that, The attribute information includes the object's hardware attributes and the object's vulnerability attributes; The object's hardware attributes include object category, object material, object position, and object color; The object's vulnerability attributes include information on whether the object is vulnerable, the maximum damage force of the object, and the robot's movement speed when the object is damaged by the robot. The safety interaction information includes whether the two corresponding objects can be combined, and the maximum safe combination force between the two objects.

3. The method according to claim 2, characterized in that, Also includes: The hardware attributes of the fourth object and the first prompt question asking whether the fourth object is vulnerable are input into the large language model to obtain the information on whether the second object is vulnerable output by the large language model. The hardware attributes of the fourth object and the second prompt question that queries the damage level of the fourth object are input into the large language model to obtain the maximum damage level of the second object output by the large language model. The object hardware attributes of the fourth object, the mass of the robot, and the third prompt question that queries the speed of the robot's movement when the fourth object is damaged by the robot are input into the large language model to obtain the speed of the robot's movement when the fourth object is damaged by the robot, which is output by the large language model. The fourth object includes any one of the first object, the second object, the third object, the first object sample, and the second object sample.

4. The method according to claim 3, characterized in that, Also includes: The identification information of each fifth object in the robot's allowed movement area is used to identify each node, and the safety interaction information between the two objects corresponding to any two nodes is used as the edge between the two nodes to construct an object safety network graph. Based on the identification information of the second object and the identification information of the third object, the security interaction information between the second object and the third object is searched in the object security network graph.

5. The method according to claim 4, characterized in that, Also includes: Using the position information of the fifth object as the center and the radius determined by the size of the machine, a corresponding circular area is determined, and each sixth object in the circular area is determined. For each of the sixth objects, the attribute information of the fifth object and the attribute information of the sixth object, as well as the fourth prompt question asking whether the fifth object and the sixth object can be combined, are input into the large language model to obtain the information output by the large language model on whether the fifth object and the sixth object can be combined. Having obtained information that the fifth object and the sixth object can be combined, a fifth prompt question is input into the large language model to inquire about the maximum safe combination strength when the fifth object and the sixth object are combined, and the maximum safe combination strength when the fifth object and the sixth object are combined is output by the large language model. Based on information about whether the fifth object and the sixth object can be combined, and the maximum safe combination strength when the fifth object and the sixth object are combined if they can be combined, the safe interaction information between the fifth object and the sixth object is determined.

6. The method according to claim 1, characterized in that, Determining the maximum instantaneous acceleration of the robot's motion based on the maximum damage force of each of the first objects and the robot's mass includes: The maximum instantaneous acceleration of the robot's motion is determined by the ratio between the average of the maximum damage force of each of the first objects and the mass of the robot.

7. A safety navigation and operation device for a robot, characterized in that, include: A triggering module is used to respond to a first instruction requesting the robot to perform an operation, triggering the robot to begin navigation and operation to execute the operation of the first instruction; The first object determination module is used to determine at least one first object located in the current area corresponding to the current position, based on the current position of the robot when the robot is moving. An acceleration determination module is used to determine the maximum instantaneous acceleration of the robot's motion based on the maximum damage force of each of the first objects and the mass of the robot. The maximum damage force refers to the critical force at which the object is damaged. If a force greater than or equal to the maximum damage force is applied to the object, the object will be damaged. The operation force determination module is used to determine the maximum operation force that the robot is allowed to operate on each of the first objects based on the maximum damage force of each of the first objects. The navigation and operation module is used to control the robot to navigate and operate based on the maximum instantaneous acceleration and the maximum operating force; The triggering module is also used for: When the first instruction is to request the robot to operate on the second object, a third object is determined based on the first instruction during the robot's operation on the second object; Based on the attribute information of the second object, the attribute information of the third object, and the safety interaction information between the second object and the third object, the execution risk of the first instruction is determined; if the execution risk meets the preset safety execution conditions, the robot is triggered to start navigation and operation. The triggering module is also used for: The first instruction, the attribute information of the second object, the attribute information of the third object, and the security interaction information between the second object and the third object are input into the security language operation model to obtain the execution risk of the first instruction output by the security language operation model. The training process of the secure language operation model includes: The first training sample is input into the secure language operation model to obtain the first risk assessment result output by the secure language operation model; The first training sample is input into the pre-trained large language model to obtain the second risk assessment result output by the large language model. Based on the first risk assessment result and the second risk assessment result, determine the loss function; Based on the loss function, adjust the model parameters of the secure language operation model; The first training sample includes an instruction sample, attribute information of a first object sample, attribute information of a second object sample, and safety interaction information between the first object sample and the second object sample. The instruction sample is an instruction requesting the robot to operate on the first object sample, and the second object sample is an object determined based on the instruction sample during the operation of the robot on the first object sample.

8. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, in, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-6.

Images