Safe navigation and operation method and device of robot and electronic equipment

Abstract

The invention provides a safe navigation and operation method and system of a robot, and relates to the technical field of artificial intelligence. According to the implementation scheme, in response to a first instruction requesting the robot to operate, the robot is triggered to start navigation and operation so as to execute the operation of the first instruction; under the condition that the robot moves, based on the current position of the robot, at least one first object located in a current area corresponding to the current position is determined; based on the maximum damage force of each first object and the mass of the robot, determining the maximum instantaneous acceleration of the motion of the robot; based on the maximum damage force of each first object, the maximum operation force of the robot allowed to operate each first object is determined; and controlling the robot to perform navigation and operation based on the maximum instantaneous acceleration and the maximum operation force.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device and electronic equipment for safe navigation and operation of a robot. Background Art

[0002] In recent years, with the rapid development of intelligent robots, robots have been widely used in many fields, especially in indoor environments, where autonomous robot navigation has become a hot topic of research. Existing robot navigation technologies mainly include visual navigation, lidar navigation, inertial navigation, etc. Among them, visual navigation is widely used due to its low cost and powerful environmental perception capabilities. Traditional visual navigation systems usually rely on image sensors to collect environmental information and achieve navigation through feature extraction and path planning algorithms. However, this method is easily affected by factors such as lighting changes and obstructions in complex environments, resulting in reduced navigation accuracy.

[0003] To solve the above problems, language navigation technology has been introduced into the field of robotics, using language commands to optimize the control of robots. However, single language navigation has problems such as poor positioning accuracy and poor environmental interactivity. Especially in complex indoor environments, the robot's understanding of language instructions is vague and imprecise due to the lack of surrounding environment information. In the process of the robot executing instructions, it is easy to have dangerous behaviors that damage surrounding objects. Therefore, how to control the robot to navigate and operate safely is a technical problem to be solved in this field. Summary of the invention

[0004] The present invention provides a robot safe navigation and operation method, device and electronic equipment, which can solve at least one of the above technical problems.

[0005] According to one aspect of the present invention, there is provided a safe navigation and operation method of a robot, comprising:

[0006] In response to a first instruction requesting the robot to perform an operation, triggering the robot to start navigation and operation to perform the operation of the first instruction;

[0007] In the case where the robot is moving, based on the current position of the robot, determining at least one first object located in a current area corresponding to the current position;

[0008] Determining a maximum instantaneous acceleration of the robot movement based on the maximum damage force of each of the first objects and the mass of the robot;

[0009] Based on the maximum damage force of each of the first objects, determining the maximum operating force that the robot is allowed to operate on each of the first objects;

[0010] Based on the maximum instantaneous acceleration and the maximum operating force, the robot is controlled to perform navigation and operation.

[0011] According to another aspect of the present invention, there is provided a safe navigation and operation device for a robot, comprising:

[0012] a trigger module, configured to trigger the robot to start navigation and operation in response to a first instruction requesting the robot to perform an operation, so as to perform the operation of the first instruction;

[0013] A first object determination module, configured to determine, when the robot is moving, based on a current position of the robot, at least one first object located in a current area corresponding to the current position;

[0014] an acceleration determination module, configured to determine a maximum instantaneous acceleration of the robot movement based on the maximum damage force of each of the first objects and the mass of the robot;

[0015] an operation force determination module, configured to determine a maximum operation force that the robot is allowed to operate on each of the first objects based on a maximum damaged force of each of the first objects;

[0016] A navigation and operation module is used to control the robot to navigate and operate based on the maximum instantaneous acceleration and the maximum operation force.

[0017] By adopting the technical solution of the present invention, in response to the first instruction requesting the robot to operate, the robot is triggered to start navigation and operation to execute the operation of the first instruction; when the robot is moving, based on the current position of the robot, at least one first object located in the current area corresponding to the current position is determined; based on the maximum damage force of each first object and the mass of the robot, the maximum instantaneous acceleration of the robot movement is determined; based on the maximum damage force of each first object, the maximum operating force that the robot is allowed to operate on each first object is determined; based on the maximum instantaneous acceleration and the maximum operating force, the robot is controlled to navigate and operate. In this way, the robot can be prevented from damaging surrounding objects as much as possible when executing instructions, thereby improving the safety performance of robot navigation and operation.

[0018] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are used to better understand the present invention and do not constitute a limitation of the present invention.

[0020] Figure 1is a flow chart of a method for safe navigation and operation of a robot according to an embodiment of the present invention;

[0021] Figure 2 is a schematic diagram of a robot control process according to an embodiment of the present invention;

[0022] Figure 3 is a structural block diagram of a safe navigation and operation device for a robot according to an embodiment of the present invention;

[0023] Figure 4 The block diagram is a block diagram of an electronic device for implementing the method according to the embodiment of the present invention. DETAILED DESCRIPTION

[0024] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be appreciated by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present invention. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0025] Figure 1 4 is a flow chart of a safe navigation and operation method of a robot according to an embodiment of the present invention.

[0026] like Figure 1 As shown, the safe navigation and operation method of the robot may include:

[0027] S110, in response to a first instruction requesting the robot to perform an operation, triggering the robot to start navigation and operation to execute the operation of the first instruction;

[0028] S120, when the robot is moving, determining, based on the current position of the robot, at least one first object located in a current area corresponding to the current position;

[0029] S130, determining a maximum instantaneous acceleration of the robot movement based on the maximum damage force of each first object and the mass of the robot;

[0030] S140, determining a maximum operating force that the robot is allowed to operate on each first object based on the maximum damaged force of each first object;

[0031] S150, based on the maximum instantaneous acceleration and the maximum operating force, controls the robot to navigate and operate.

[0032] Exemplarily, the first instruction may be a voice instruction or a text instruction, or a voice text instruction obtained by performing voice recognition on the voice instruction.

[0033] Exemplarily, the first instruction may be an instruction requesting the robot to operate on one or more objects, or may be an instruction requesting the robot to operate on itself.

[0034] Exemplarily, the first object may include an object in the first instruction.

[0035] For example, Figure 2 As shown, the operation provides voice instructions to the visual language navigation device in the mobile robot control system through the control terminal, and the visual language navigation device recognizes the voice instructions and executes the method provided by the embodiment of the present invention, and then provides the obtained information such as the maximum instantaneous acceleration and maximum operating force of the robot, and the attribute information of the surrounding objects to the computing terminal, and the computing terminal provides navigation point information to the mobile robot control system, and the mobile robot control system controls the movement and operation of the mobile robot body through mobile instructions.

[0036] For example, the current area may be a circular area with the robot as the center and within a set radius. The radius may be the working radius of the robot. The object located in the circular area may be one or more objects.

[0037] For example, the maximum damage force can be considered as the critical force when the object is damaged. If the force or a force greater than the force is applied to the object, the object will be damaged; if a force less than the force is applied to the object, the object will not be damaged.

[0038] Exemplarily, a minimum value is determined from the maximum damage forces of the first objects, and the ratio between the minimum value and the mass of the robot is used as the maximum instantaneous acceleration of the robot's movement.

[0039] Exemplarily, the maximum damage forces of the first objects are averaged to obtain an average value, and the ratio between the average value and the mass of the robot is used as the maximum instantaneous acceleration of the robot's movement.

[0040] In this way, even if the robot collides with these objects during movement, these objects will not be damaged.

[0041] Exemplarily, if the first instruction is to request the robot to operate the second object, the maximum operating force allowed for the robot on each first object and the second object is determined based on the maximum damaged force of each first object and the maximum damaged force of the second object.

[0042] For example, the minimum value of the maximum damage force of each first object and the maximum damage force of the second object is used as the maximum operation force of the robot. For another example, since the robot operates the second object, the maximum damage force of the second object can be used as the maximum operation force of the robot.

[0043] For another example, based on the first instruction, determine the navigation route planned by the robot to execute the first instruction, based on the navigation route, determine the target object that the robot will contact among each first object, and use the minimum value of the maximum damage force of the target object and the maximum damage force of the second object as the maximum operating force of the robot.

[0044] It can be understood that the second object is different from each of the first objects, or the second object is one of the first objects.

[0045] Exemplarily, the robot is controlled to move according to the navigation path based on the maximum instantaneous acceleration.

[0046] Continuing with the previous example, when the robot reaches the target position and starts to operate the second object, the robot operates the second object based on the maximum operating force.

[0047] Exemplarily, if the first instruction is to request the robot to combine the second object with the third object, the maximum operating force of the robot to operate the second object is determined based on the maximum damage force of the second object and the maximum safe combined force of the second object and the third object. The operation can be a thrust or a throw, or a clamping force, or two of these three forces. For example, the robot first clamps the second object, and then pushes, pulls, or throws the second object to combine the second object with the third object. The maximum damage force of the second object is used as the clamping force of the robot, and the maximum safe combined force of the second object and the third object is used as the thrust or throw.

[0048] Exemplarily, combining the second object with the third object may be the following examples: pushing a chair into a table, placing a bowl on a tabletop, and placing a basketball into a basket, etc.

[0049] Exemplarily, the maximum safe combined force of the combination of the second object and the third object can be understood as, when the second object and the third object are combined with the maximum safe combined force and a force less than this force, the second object and the third object are not damaged; when the second object and the third object are combined with a force greater than the maximum safe combined force, the second object and the third object are damaged.

[0050] According to the above embodiment, in response to the first instruction requesting the robot to perform an operation, the robot is triggered to start navigation and operation to perform the operation of the first instruction; when the robot is moving, based on the current position of the robot, at least one first object located in the current area corresponding to the current position is determined; based on the maximum damage force of each first object and the mass of the robot, the maximum instantaneous acceleration of the robot movement is determined; based on the maximum damage force of each first object, the maximum operating force that the robot is allowed to operate on each first object is determined; based on the maximum instantaneous acceleration and the maximum operating force, the robot is controlled to perform navigation and operation. In this way, the robot can be prevented from damaging surrounding objects as much as possible when executing instructions, thereby improving the safety performance of robot navigation and operation.

[0051] In one embodiment, the above-mentioned step of triggering the robot to start navigation and operation includes: when the first instruction is to request the robot to operate on the second object, based on the first instruction, determining the third object involved in the process of the robot operating the second object; based on the attribute information of the second object, the attribute information of the third object, and the security interaction information between the second object and the third object, determining the execution risk of the first instruction; when the execution risk meets the preset security execution conditions, triggering the robot to start navigation and operation.

[0052] Exemplarily, the first instruction is to request the robot to operate the second object, including requesting the robot to operate only the second object, and requesting the robot to combine the second object with a third object.

[0053] Exemplarily, the third object may be an object to which the second object is combined, or may be an object on which the robot needs to operate the third object before operating the second object.

[0054] Exemplarily, the attribute information of the second object may include object hardware attributes and object fragility attributes of the second object.

[0055] Exemplarily, the attribute information of the third object may include object hardware attributes and object fragility attributes of the third object.

[0056] The hardware attributes of the object include object category, object material, object position and object color. The fragile attributes of the object include information about whether the object is fragile, the maximum damage strength of the object, and the movement speed of the robot when the object is damaged by the robot.

[0057] Exemplarily, the safety interaction information between the second object and the third object includes whether the second object and the third object can be combined, and if they can be combined, the maximum safety combination strength of the second object and the third object.

[0058] Exemplarily, a pre-trained model may be used to process the three pieces of information and instructions to obtain the execution risk of the first instruction.

[0059] Exemplarily, the object fragility property of the second object, the object fragility property of the third object, and the maximum safety combined strength of the second object and the third object can be used for mapping processing respectively, and then the three mapping results are weightedly summed to obtain the execution risk of the first instruction.

[0060] Exemplarily, the safe execution condition may be that the execution risk is less than a preset risk threshold. If the execution risk is less than the preset risk threshold, the robot is triggered to start navigation and operation. If the execution risk is greater than the preset risk threshold, a warning indication of refusing to execute the instruction is issued. The warning indication may include the above-mentioned execution risk.

[0061] In this example, the risk of the instruction is evaluated by combining the object's attribute label, fragile label, and safety interaction strength. If the instruction may cause the object to be damaged, the robot will issue a warning and immediately execute the task corresponding to the instruction. This improves the safety performance of the robot's navigation and operation.

[0062] In one embodiment, the above-mentioned determination of the execution risk of the first instruction based on the attribute information of the second object, the attribute information of the third object, and the security interaction information between the second object and the third object includes: inputting the first instruction, the attribute information of the second object, the attribute information of the third object, and the security interaction information between the second object and the third object into a security language operation model, and obtaining the execution risk of the first instruction output by the security language operation model.

[0063] Among them, the training process of the safe language operation model includes: inputting the first training sample into the safe language operation model to obtain a first risk assessment result output by the safe language operation model; inputting the first training sample into the trained large language model to obtain a second risk assessment result output by the large language model; determining a loss function based on the first risk assessment result and the second risk assessment result; and adjusting the model parameters of the safe language operation model based on the loss function.

[0064] Among them, the first training sample includes an instruction sample, attribute information of the first object sample, attribute information of the second object sample and security interaction information between the first object sample and the second object sample. The instruction sample is an instruction requesting the robot to operate the first object sample, and the second object sample is an object involved in the process of the machine operating the first object sample based on the instruction sample.

[0065] Exemplarily, the first instruction, the attribute information of the second object, the attribute information of the third object, and the security interaction information between the second object and the third object are input into the security language operation model. The security language operation model extracts features of the type, material, location and other information of the object through a feature extraction module (such as CNN or Transformer) to obtain corresponding feature representations. At the same time, the security language operation model uses a natural language processing (NLP) module to parse the first instruction to obtain the key verbs in the instruction and the description of the object. Then, the security language operation model uses the extracted features and the parsed information to predict the execution risk of the first instruction.

[0066] For example, a scoring value may be used to represent the execution risk, and the higher the execution risk, the higher the scoring value.

[0067] Understandably, a large language model (LLM) is used to infer the potential impact of the instruction to determine whether operating the instruction sample will cause damage to the object. That is, the second risk assessment result is used as the true value data or labeled data in the model training process. The first risk assessment result is the predicted data in the model training process. In this way, the training data used in the training process does not need to be manually labeled to obtain the true value data, and the large language model is used to obtain the true value data, which is conducive to improving the labeling efficiency and thus improving the training speed.

[0068] According to the above-mentioned implementation, a safe language operation model can be trained accurately and quickly, and the safe language operation model can be used to process the first instruction, the attribute information of the second object, the attribute information of the third object, and the safe interaction information between the second object and the third object to obtain the execution risk of the first instruction.

[0069] In one embodiment, the attribute information includes object hardware attributes and object vulnerability attributes; the object hardware attributes include object category, object material, object position and object color; the object vulnerability attributes include information on whether the object is fragile, the maximum damage strength of the object, and the movement speed of the robot when the object is damaged by the robot; the safety interaction information includes whether the corresponding two objects can be combined and the maximum safety combination strength between the two objects.

[0070] Exemplarily, the object hardware attribute of any fourth object may be determined in the following manner:

[0071] The RAM model and the YOLO model are used to process the RGB image including the fourth object to obtain a first classification result and a second classification result of the fourth object; the material estimation model is used to process the RGB image including the fourth object to obtain the object material of the fourth object; a back projection calculation is performed on the RGB image including the fourth object and a depth image corresponding to the image to obtain the object position of the fourth object; color extraction is performed on the RGB image including the fourth object to obtain the object color of the fourth object; the first classification result and the second classification result, the object material, the object position and the object color of the fourth object are input into the multimodal classification model to obtain the object category of the fourth object output by the multimodal classification model.

[0072] Exemplarily, the multimodal classification model includes a multimodal feature extraction module, a feature fusion module and a classification module.

[0073] For example, the multimodal feature extraction module uses different sub-networks to perform specific extraction on each feature of the object. For example, a convolutional neural network (CNN) is used to extract RGB image features, a PointNet or Transformer network is used to extract point cloud features, and a fully connected layer is used to process label confidence scores and material features.

[0074] Exemplarily, the feature fusion module integrates features of different modalities into a unified feature representation through concatenation, attention mechanism or other fusion strategies.

[0075] Exemplarily, the classification module: after fusing the features, the score of each object label is output through a fully connected network for selecting the optimal label.

[0076] According to the above example, a multimodal classification model is used to take the two classification results of the object, the object material, the object spatial pose matrix, the object volume, and the object color as input, score the label of each object, and select the label with the highest score as the final label of the object.

[0077] Exemplarily, the object category of each object can be calculated according to preset steps to construct a sliding window sequence, save the object category label at each moment, and select the label with the most occurrences as the current final object category of the object, so as to ensure the accuracy of the object category.

[0078] According to the above implementation, the object hardware properties of the object can be accurately calculated.

[0079] In one embodiment, the above method further comprises:

[0080] Using the identification information of each fifth object in the robot's allowed motion area as identification for each node, using the safety interaction information between two objects corresponding to any two nodes as the edge between the two nodes, to construct an object safety network graph;

[0081] Based on the identification information of the second object and the identification information of the third object, security interaction information between the second object and the third object is searched in the object security network map.

[0082] In one embodiment, the method further includes: inputting the object hardware properties of the fourth object and a first prompt question inquiring whether the fourth object is fragile into the large language model, and obtaining information on whether the second object is fragile output by the large language model; inputting the object hardware properties of the fourth object and a second prompt question inquiring about the degree of damage to the fourth object into the large language model, and obtaining the maximum degree of damage to the second object output by the large language model; inputting the object hardware properties of the fourth object, the mass of the robot, and a third prompt question inquiring about the movement speed of the machine when the fourth object is damaged by the machine into the large language model, and obtaining the movement speed of the machine when the fourth object is damaged by the machine output by the large language model; wherein the fourth object includes any one of the first object, the second object, the third object, the first object sample, and the second object sample.

[0083] Exemplarily, the fourth object may be an object in the allowed movement area of ​​the robot.

[0084] Exemplarily, each object in the robot's allowed motion area is used as a node, and the attribute information of each object is used as node information, and the safety interaction information between two objects corresponding to any two nodes is used as the edge between the two nodes, so as to construct an object safety network diagram.

[0085] Exemplarily, the attribute information of each object can be obtained from the object safety network diagram.

[0086] For example, after obtaining the attribute information of each object, a large language model (LLM) is used to perform a risk analysis on each object. Specifically, the LLM is asked the following questions:

[0087] The first prompt question is, for example: "Is the material of this object fragile?".

[0088] The second prompt question is, for example: "How much force (in Newtons) will cause damage to the object (defined as the object's safe interaction force or maximum damaging force)".

[0089] The third prompt question is, for example: "The mass of the robot is X kg. In the current scenario, at what speed will the robot cause damage to the object?".

[0090] Thus, according to the above answers, three fragile labels are generated for each object: whether it is fragile, the maximum damage force (maximum safe interaction force) and the maximum safe speed.

[0091] According to the above-mentioned embodiment, the object vulnerability attribute of each object is generated using a large language model.

[0092] In one embodiment, the method further includes: determining a corresponding circular area with the position information of the fifth object as the center and a radius determined by the size of the machine, and determining each sixth object in the circular area; for each sixth object, inputting the attribute information of the fifth object and the attribute information of the sixth object, as well as a fourth prompt question inquiring whether the fifth object and the sixth object can be combined into the large language model, and obtaining information output by the large language model on whether the fifth object and the sixth object can be combined; when the information that the fifth object and the sixth object can be combined is obtained, inputting the fifth prompt question inquiring about the maximum safe combination strength when the fifth object and the sixth object are combined into the large language model, and obtaining the maximum safe combination strength when the fifth object and the sixth object are combined output by the large language model; based on the information on whether the fifth object and the sixth object can be combined, and the maximum safe combination strength when the fifth object and the sixth object are combined when the fifth object and the sixth object can be combined, determine the safety interaction information of the fifth object and the sixth object.

[0093] Exemplarily, each object is represented as a node in the object safety network graph, and the node information includes the object's location, three fragile labels, material, color and other attributes.

[0094] Exemplarily, with each object as the center (referred to as object A), objects within the circumference are determined with a radius twice the maximum width of the robot's body (referred to as object B).

[0095] For example, for object A and object B, LLM query is used:

[0096] The fourth prompt question is, for example, “Can these two objects be combined or interacted with?” (e.g., push a chair to the table, place a bowl on the table, put a basketball into a basket).

[0097] If the combination is possible, a fifth prompt question is asked, such as: "What is the maximum safe combination force when combined?" (Unit: Newton).

[0098] Through the above steps, a safe interaction edge connecting object A and object B is generated, which contains two key information: whether they can be combined, and the maximum safe combination force (the force that cannot be exceeded during combination, unit: Newton).

[0099] According to the above implementation, the edges between the nodes in the object safety network graph can be constructed.

[0100] In one embodiment, determining the maximum instantaneous acceleration of the robot's movement based on the maximum damage force of each first object and the mass of the robot may include: determining the maximum instantaneous acceleration of the robot's movement based on the ratio between the average value of the maximum damage force of each first object and the mass of the robot.

[0101] According to the above embodiment, when the robot moves at the maximum instantaneous acceleration, even if it collides with surrounding objects, it will not damage the surrounding objects.

[0102] Figure 3 It is a structural block diagram of a safe navigation and operation device for a robot according to an embodiment of the present invention.

[0103] like Figure 3 As shown, the robot's safe navigation and operation device may include:

[0104] A trigger module 310 is used to trigger the robot to start navigation and operation in response to a first instruction requesting the robot to perform an operation, so as to perform the operation of the first instruction;

[0105] A first object determination module 320, configured to determine, when the robot is moving, based on the current position of the robot, at least one first object located in a current area corresponding to the current position;

[0106] An acceleration determination module 330, configured to determine a maximum instantaneous acceleration of the robot movement based on the maximum damage force of each of the first objects and the mass of the robot;

[0107] An operation force determination module 340, configured to determine a maximum operation force that the robot is allowed to operate on each of the first objects based on the maximum damage force of each of the first objects;

[0108] The navigation and operation module 350 is used to control the robot to navigate and operate based on the maximum instantaneous acceleration and the maximum operation force.

[0109] In one implementation, the trigger module 310 includes:

[0110] A second object determining unit, configured to determine, based on the first instruction, a third object involved in the process of the robot operating the second object when the first instruction is a request for the robot to operate the second object;

[0111] an execution risk determination unit, configured to determine an execution risk of the first instruction based on the attribute information of the second object, the attribute information of the third object, and the security interaction information between the second object and the third object;

[0112] The triggering working unit is used to trigger the robot to start navigation and operation when the execution risk meets the preset safe execution conditions.

[0113] In one implementation, the execution risk determination unit is specifically configured to:

[0114] Inputting the first instruction, the attribute information of the second object, the attribute information of the third object, and the security interaction information between the second object and the third object into a security language operation model, and obtaining the execution risk of the first instruction output by the security language operation model;

[0115] The training process of the secure language operation model includes:

[0116] Inputting a first training sample into the secure language operation model to obtain a first risk assessment result output by the secure language operation model;

[0117] Inputting the first training sample into a trained large language model to obtain a second risk assessment result output by the large language model;

[0118] Determining a loss function based on the first risk assessment result and the second risk assessment result;

[0119] Based on the loss function, adjusting model parameters of the secure language operation model;

[0120] Among them, the first training sample includes an instruction sample, attribute information of the first object sample, attribute information of the second object sample and security interaction information between the first object sample and the second object sample. The instruction sample is an instruction requesting the robot to operate the first object sample, and the second object sample is an object involved in the process of the machine operating the first object sample based on the instruction sample.

[0121] In one embodiment, the attribute information includes object hardware attributes and object fragility attributes;

[0122] The object hardware attributes include object category, object material, object position and object color;

[0123] The object damage attribute includes information on whether the object is damageable, the maximum damage strength of the object, and the movement speed of the robot when the object is damaged by the robot;

[0124] The safety interaction information includes whether the corresponding two objects can be combined and the maximum safety combination strength between the two objects.

[0125] In one embodiment, the above device further comprises:

[0126] A first fragile information determination module is used to input the object hardware attribute of the fourth object and a first prompt question asking whether the fourth object is fragile into the large language model, and obtain information on whether the second object is fragile output by the large language model;

[0127] A second damage information determination module is configured to input the object hardware attribute of the fourth object and a second prompt question regarding the damage intensity of the fourth object into the large language model, and obtain the maximum damage intensity of the second object output by the large language model;

[0128] a third fragile information determination module, configured to input the object hardware attribute of the fourth object, the mass of the robot, and a third prompt question asking about the movement speed of the machine when the fourth object is damaged by the machine into the large language model, and obtain the movement speed of the machine when the fourth object is damaged by the machine output by the large language model;

[0129] The fourth object includes any one of the first object, the second object, the third object, the first object sample, and the second object sample.

[0130] In one embodiment, the above device further comprises:

[0131] A network graph construction module, used to identify each node by using the identification information of each fifth object in the allowed motion area of ​​the robot, and to construct an object safety network graph by using the safety interaction information between two objects corresponding to any two nodes as the edge between the two nodes;

[0132] The side information search module is used to search for the security interaction information between the second object and the third object in the object security network diagram based on the identification information of the second object and the identification information of the third object.

[0133] In one embodiment, the above device further comprises:

[0134] an area determination module, used to determine a corresponding circular area with the position information of the fifth object as the center and a radius determined by the size of the machine, and to determine each sixth object in the circular area;

[0135] a first combination information determination module, for each of the sixth objects, inputting the attribute information of the fifth object and the attribute information of the sixth object, and a fourth prompt question asking whether the fifth object and the sixth object can be combined into the large language model, and obtaining information output by the large language model on whether the fifth object and the sixth object can be combined;

[0136] a second combination information determination module, for inputting a fifth prompt question regarding the maximum safe combination strength when the fifth object and the sixth object are combined into the large language model, upon obtaining information that the fifth object and the sixth object can be combined, to obtain the maximum safe combination strength when the fifth object and the sixth object are combined output by the large language model;

[0137] The interaction information determination module is used to determine the safety interaction information of the fifth object and the sixth object based on information on whether the fifth object and the sixth object can be combined, and the maximum safe combination strength when the fifth object and the sixth object are combined if the fifth object and the sixth object can be combined.

[0138] In one implementation, the acceleration determination module 330 is specifically used to determine the maximum instantaneous acceleration of the robot movement based on the ratio between the average value of the maximum damage force of each of the first objects and the mass of the robot.

[0139] For the description of specific functions and examples of each module and submodule of the system in the embodiment of the present invention, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0140] In the technical solution of the present invention, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0141] According to an embodiment of the present invention, the present invention also provides a system and a readable storage medium.

[0142] Figure 4 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0143] like Figure 4 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0144] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0145] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the safe navigation and operation method of the robot. For example, in some embodiments, the safe navigation and operation method of the robot may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the safe navigation and operation method of the robot described above may be executed. Alternatively, in other embodiments, the computing unit 801 may be configured to execute the safe navigation and operation method of the robot in any other appropriate manner (e.g., by means of firmware).

[0146] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0147] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0148] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0149] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0150] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0151] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0152] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document does not limit this.

[0153] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A safe navigation and operation method for a robot, characterized in that: include: In response to a first instruction requesting the robot to perform an operation, triggering the robot to start navigation and operation to perform the operation of the first instruction; In the case where the robot is moving, based on the current position of the robot, determining at least one first object located in a current area corresponding to the current position; Determining a maximum instantaneous acceleration of the robot movement based on the maximum damage force of each of the first objects and the mass of the robot; Based on the maximum damage force of each of the first objects, determining the maximum operating force that the robot is allowed to operate on each of the first objects; Based on the maximum instantaneous acceleration and the maximum operating force, the robot is controlled to perform navigation and operation.

2. The method according to claim 1, characterized in that The triggering the robot to start navigation and operation includes: In a case where the first instruction is a request for the robot to operate a second object, determining, based on the first instruction, a third object involved in the process of the robot operating the second object; Determining an execution risk of the first instruction based on the attribute information of the second object, the attribute information of the third object, and security interaction information between the second object and the third object; When the execution risk meets the preset safe execution conditions, the robot is triggered to start navigation and operation.

3. The method according to claim 2, characterized in that The determining the execution risk of the first instruction based on the attribute information of the second object, the attribute information of the third object, and the security interaction information between the second object and the third object includes: Inputting the first instruction, the attribute information of the second object, the attribute information of the third object, and the security interaction information between the second object and the third object into a security language operation model, and obtaining the execution risk of the first instruction output by the security language operation model; The training process of the secure language operation model includes: Inputting a first training sample into the secure language operation model to obtain a first risk assessment result output by the secure language operation model; Inputting the first training sample into a trained large language model to obtain a second risk assessment result output by the large language model; Determining a loss function based on the first risk assessment result and the second risk assessment result; Based on the loss function, adjusting model parameters of the secure language operation model; Among them, the first training sample includes an instruction sample, attribute information of the first object sample, attribute information of the second object sample and security interaction information between the first object sample and the second object sample. The instruction sample is an instruction requesting the robot to operate the first object sample, and the second object sample is an object involved in the process of the machine operating the first object sample based on the instruction sample.

4. The method according to claim 3, characterized in that The attribute information includes object hardware attributes and object vulnerability attributes; The object hardware attributes include object category, object material, object position and object color; The object damage attribute includes information on whether the object is damageable, the maximum damage strength of the object, and the movement speed of the robot when the object is damaged by the robot; The safety interaction information includes whether the corresponding two objects can be combined and the maximum safety combination strength between the two objects.

5. The method according to claim 4, characterized in that Also includes: Inputting the object hardware attribute of the fourth object and the first prompt question asking whether the fourth object is fragile into the large language model, and obtaining information of whether the second object is fragile output by the large language model; Inputting the object hardware attribute of the fourth object and the second prompt question regarding the damage strength of the fourth object into the large language model, and obtaining the maximum damage strength of the second object output by the large language model; Inputting the object hardware attribute of the fourth object, the mass of the robot, and a third prompt question asking about the movement speed of the machine when the fourth object is damaged by the machine into the large language model, and obtaining the movement speed of the machine when the fourth object is damaged by the machine output by the large language model; The fourth object includes any one of the first object, the second object, the third object, the first object sample, and the second object sample.

6. The method according to claim 4, characterized in that Also includes: Using identification information of each fifth object in the allowed motion area of ​​the robot as identification for each node, using safety interaction information between two objects corresponding to any two nodes as an edge between the two nodes, to construct an object safety network graph; Based on the identification information of the second object and the identification information of the third object, security interaction information between the second object and the third object is searched in the object security network map.

7. The method according to claim 6, characterized in that Also includes: Determine a corresponding circular area with the position information of the fifth object as the center and a radius determined by the size of the machine, and determine each sixth object in the circular area; For each of the sixth objects, the attribute information of the fifth object and the attribute information of the sixth object, as well as a fourth prompt question asking whether the fifth object and the sixth object can be combined, are respectively input into the large language model, so as to obtain information output by the large language model on whether the fifth object and the sixth object can be combined; When information that the fifth object and the sixth object can be combined is obtained, a fifth prompt question for inquiring about the maximum safe combination strength when the fifth object and the sixth object are combined is input into the large language model, and the maximum safe combination strength when the fifth object and the sixth object are combined is obtained as output by the large language model; Based on the information of whether the fifth object and the sixth object can be combined, and the maximum safe combination strength when the fifth object and the sixth object are combined if the fifth object and the sixth object can be combined, the safe interaction information of the fifth object and the sixth object is determined.

8. The method according to any one of claims 1 to 7, characterized in that The determining the maximum instantaneous acceleration of the robot movement based on the maximum damage force of each of the first objects and the mass of the robot comprises: The maximum instantaneous acceleration of the robot movement is determined based on the ratio between the average value of the maximum damage forces of the first objects and the mass of the robot.

9. A robot safe navigation and operation device, characterized in that: include: a trigger module, configured to trigger the robot to start navigation and operation in response to a first instruction requesting the robot to perform an operation, so as to perform the operation of the first instruction; A first object determination module, configured to determine, when the robot is moving, based on a current position of the robot, at least one first object located in a current area corresponding to the current position; an acceleration determination module, configured to determine a maximum instantaneous acceleration of the robot movement based on the maximum damage force of each of the first objects and the mass of the robot; an operation force determination module, configured to determine a maximum operation force that the robot is allowed to operate on each of the first objects based on a maximum damaged force of each of the first objects; A navigation and operation module is used to control the robot to navigate and operate based on the maximum instantaneous acceleration and the maximum operation force.

10. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

11. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-7.

Images