Scene risk guidance-based intelligent fuzzy language instruction identification method

Abstract

The invention relates to the technical field of ownership intelligence, in particular to a ownership intelligence fuzzy language instruction recognition method based on scene risk guidance. In order to solve the problem that a fuzzy language instruction recognition method in the prior art easily causes understanding deviation and potential safety hazards, the invention provides an intelligent fuzzy language instruction recognition method based on scene risk guidance, which comprises the following steps of: 1) constructing a scene risk map according to a basic risk and a distance risk; 2) identifying an instruction type; 3) instruction analysis and risk assessment; and 4) instruction rewriting and object identification replacement. According to the method, a risk perception mechanism is introduced, the high-risk behavior execution probability caused by semantic deviation of the intelligent robot is effectively reduced, the understanding accuracy and interaction safety of the intelligent robot in a real scene are improved, and the method is suitable for a complex application environment with high requirements for task reliability and behavior controllability of the intelligent robot.

Description

Technical Field

[0001] This invention relates to the field of embodied intelligence technology, specifically to an embodied intelligence fuzzy language instruction recognition method based on scenario risk guidance. Background Technology

[0002] In recent years, with the development of large-scale models, embodied large-scale models have gradually become the core capability support for embodied intelligence. Embodied large-scale models mainly include two categories: end-to-end models and hierarchical models. The former, based on the Visual-Language-Motion (VLA) model, can directly map perceptual inputs such as vision and language to action outputs, with representative achievements such as RT-2 and OpenVLA; the latter, based on the Visual-Language (VLM) model, adopts a "brain-cerebellum" hierarchical structure, separating abstract task planning from low-level action control, and has stronger real-time performance and module adaptability, such as Zhiyuan GO-1, Figure 01, and Rekep. Regardless of the structure, multimodal modeling, reinforcement learning, knowledge injection, and large-scale training have become the mainstream technical paths for improving the "perception-decision-execution" capabilities of intelligent agents. However, in actual interaction, user natural language commands often have ambiguity, such as unclear command expression, incomplete target description, or unclear context. Although such ambiguous language commands are not directly dangerous, they are prone to semantic comprehension bias, which can lead to incorrect actions or unsafe behaviors in the physical world. For example, while the instruction "Please put the knife in a safe place" clearly expresses a safety intention, its language remains ambiguous. If embodied intelligence relies solely on literal language and visual features for understanding, it may misunderstand "safety" as "invisible" or "unaware of others," thus placing the knife in locations such as behind a cabinet or in a furniture crevic. While these actions may seem reasonable on the surface, they could contradict the user's actual intentions and create potential safety hazards.

[0003] Although existing methods have proposed various coping strategies, most current methods for handling fuzzy language instructions aim to improve task completion rates, focusing primarily on language disambiguation, task reconstruction, or user interaction experience. They fail to fully incorporate safety factors such as object relationships, contact risks, and spatial limitations in the parsing process, thus making it difficult to meet the safety requirements of embodied intelligence in real-world scenarios. Summary of the Invention

[0004] In order to address the issues that existing methods for recognizing fuzzy language instructions are prone to misunderstandings and security risks, this invention provides an embodied intelligent fuzzy language instruction recognition method based on scenario risk guidance.

[0005] This invention is achieved using the following technical solution:

[0006] A method for recognizing embodied intelligent fuzzy language instructions based on scenario risk guidance includes the following steps:

[0007] 1) Construct a scene risk map: Divide the risks between objects in the scene into two categories. One category is the basic risks brought about by the inherent characteristics or potential interactions of the objects, and the other category is the impact of the physical distance between the objects on these potential risks, i.e., distance risks.

[0008] ① Basic Risk: The Large Language Model (LLM) infers and generates corresponding risk assessment values ​​based on the attributes, types, and relative relationships of objects. The basic risk is then divided into five levels according to these risk assessment values:

[0009] a. Extremely low risk, basic risk assessment value (0, 0.2]: indicates that the risk is very low, and even if contact occurs, it is almost impossible to cause harm;

[0010] b. Lower risk, basic risk assessment value A value of (0.2, 0.4) indicates a low risk; while there is a possibility of minor harm, it is generally acceptable, but appropriate caution is still required.

[0011] c. Medium risk, basic risk assessment value A value of (0.4, 0.6) indicates that there is a certain risk; improper handling may result in damage to the item or minor personal injury.

[0012] d. High risk, basic risk assessment value A value of (0.6, 0.8) indicates a high probability of injury, which may have serious consequences for personnel or equipment, requiring immediate preventive measures.

[0013] e. Extremely high risk, basic risk assessment value (0.8, 1.0]: This indicates a serious risk; the coexistence of the two objects will almost certainly lead to catastrophic consequences, and immediate intervention is necessary.

[0014] ② Distance risk: Assume the position coordinates of object A are... The position coordinates of object B are The formula for calculating the Euclidean distance d between them is as follows:

[0015] ;

[0016] When the Euclidean distance d between objects is greater than the safety threshold When the distance is zero, the risk is zero; however, when the Euclidean distance between objects is less than the safety threshold... In this case, the distance risk is normalized to the [0, 1] interval to unify the differences in distance scales across different scenarios and ensure that distance risks are comparable across different spatial scales. The distance risk formula is as follows:

[0017]

[0018] in, For distance risk; It is the attenuation coefficient, used to control the rate of attenuation; It is the base of the natural logarithm;

[0019] ③ Calculation of comprehensive risk between objects: The comprehensive risk between objects can be obtained based on the basic risk and the distance risk. The calculation formula is as follows:

[0020] ;

[0021] ④ Construct a scene risk graph: treat each object in the scene as a node in the graph. The overall risk value between object pairs For border rights Construct a weighted undirected graph This forms the final "scenario risk map";

[0022] 2) Identify command type: Input user language commands and object information detected in the current scene, call the multimodal model to perform preliminary semantic analysis, and classify them into the following three categories according to the clarity of the command content:

[0023] a. Instructions that are too vague to be parsed: The language instructions contain pronouns that cannot be inferred or ambiguous references, making it impossible to determine the specific target;

[0024] b. Vague but clear instructions: Although there is semantic ambiguity, the intention can be clarified through contextual information or common sense reasoning;

[0025] c. Clear instructions: The language instructions clearly express the task objectives and the objects to be operated on;

[0026] 3) Instruction analysis and risk assessment:

[0027] ① The instruction is too vague to be parsed: it does not infer the target object or behavior;

[0028] ② Ambiguous but clear instructions: Use a multimodal model to generate several possible clear candidate instructions, combine spaCy natural language processing tools to extract the key objects involved in each instruction, find their risk relationships in the scene risk map, calculate the average risk score of each candidate instruction, and select the one with the lowest risk as the final execution instruction;

[0029] ③ Clear instructions: Retain the user's original instructions as the final execution instructions without modification;

[0030] 4) Instruction rewriting and object identifier replacement:

[0031] The final execution instruction that has been judged or processed is rewritten and the object identifier is replaced, specifically including the following steps:

[0032] ① Identify the optimal target object: When there are multiple objects of the same type or name in the scene, compare and analyze them in conjunction with the scene risk map, and select the object that is relevant to the task and has the lowest risk value as the final target to reduce the execution risk;

[0033] ② Instruction text rewriting: Replace the natural language object names involved in the original instructions with the corresponding unique node numbers in the risk map, thereby realizing the conversion from natural language instructions to structured instructions;

[0034] ③ Output standard format: The output standard format adopts a structured form, which makes it easy for downstream modules to directly call and execute.

[0035] Furthermore, the output standard format is JSON.

[0036] Furthermore, LLM specifically adopts the GPT-4o model.

[0037] The beneficial effects of this invention are as follows: By constructing a scene risk map that integrates object attributes and spatial relationships, this invention guides a large language model to comprehensively consider task objectives and environmental risks during instruction parsing. Without changing the user's original language input, it achieves semantic clarification and rewriting of fuzzy instructions from embodied intelligence. Compared with traditional language parsing methods that rely solely on context or external knowledge, this invention introduces a risk perception mechanism, which effectively reduces the probability of high-risk behavior execution caused by semantic deviations in embodied intelligence, improves its understanding accuracy and interaction security in real-world scenarios, and is suitable for complex application environments with high requirements for the reliability and controllability of embodied intelligence tasks. Attached Figure Description

[0038] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a schematic diagram of the method described in this invention. Detailed Implementation

[0041] To better understand the above-mentioned objectives, features, and advantages of the present invention, the solutions of the present invention will be further described below. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0042] In this description, it should be noted that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. It should also be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joint" should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.

[0043] Many specific details are set forth in the following description in order to provide a full understanding of the invention, but the invention may also be practiced in other ways different from those described herein; obviously, the embodiments in the specification are only some embodiments of the invention, and not all embodiments.

[0044] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0045] A method for recognizing embodied intelligent fuzzy language instructions based on scenario risk guidance includes the following steps:

[0046] 1) Constructing a Scene Risk Map: In embodied intelligent operating scenarios, risks not only stem from the inherent dangerous properties of objects but are also influenced by their spatial location and potential interactions with other objects. Even with the same hazard source, the actual threat level can vary significantly under different spatial layouts. For example, the risk of a sharp tool being close to the operating area is significantly higher than being far away. Therefore, the risks between objects in the scene are divided into two categories: one is the basic risk arising from the inherent characteristics or potential interactions of the objects, and the other is the impact of the physical distance between objects on these potential risks, i.e., distance risk.

[0047] ① Basic Risk: The Large Language Model (LLM) (in practice, the model generation uses the GPT-4o model, and the later verification and validation of the model uses Deepseek and Qwen; the risk assessment values ​​generated by the GPT-4o model have been verified and are usable) infers and generates corresponding risk assessment values ​​based on the attributes, types, and relative relationships of objects. The basic risk is then divided into the following five levels according to the risk assessment values:

[0048] a. Extremely low risk, basic risk assessment value (0, 0.2]: indicates that the risk is very low, and even if contact occurs, it is almost impossible to cause harm. For example, a book and a pillow are both relatively soft and will not cause harm to the body.

[0049] b. Lower risk, basic risk assessment value (0.2, 0.4) indicates a low risk. Although there is a possibility of minor injury, it is generally acceptable, but appropriate care is still required. For example, a plastic toy on a carpet may cause a minor trip but will not cause serious injury.

[0050] c. Medium risk, basic risk assessment value (0.4, 0.6) indicates that there is a certain risk. If not handled properly, it may result in damage to items or minor personal injury. For example, a hot cup placed near the papers may be knocked over and damage important documents.

[0051] d. High risk, basic risk assessment value A value of (0.6, 0.8) indicates a high probability of injury, which may have serious consequences for people or equipment and requires immediate preventive measures. For example, metal utensils near power outlets may cause electric shock.

[0052] e. Extremely high risk, basic risk assessment value (0.8, 1.0) indicates a serious risk; the coexistence of the two objects will almost certainly lead to catastrophic consequences and requires immediate intervention. For example, flammable materials near a working gas stove can easily cause a fire and result in serious damage.

[0053] ② Distance Risk: By using an exponential decay function to quantify distance risk, we can better simulate the interactions between objects in real life. Assume the position coordinates of object A are... The position coordinates of object B are The formula for calculating the Euclidean distance d between them is as follows:

[0054] ;

[0055] When the Euclidean distance d between objects is greater than the safety threshold When the distance is zero, the risk is zero; however, when the Euclidean distance between objects is less than the safety threshold... In this case, the distance risk is normalized to the [0, 1] interval to unify the differences in distance scales across different scenarios and ensure that distance risks are comparable across different spatial scales. The distance risk formula is as follows:

[0056]

[0057] in, For distance risk; It is the attenuation coefficient, used to control the rate of attenuation; It is the base of the natural logarithm;

[0058] ③ Calculation of comprehensive risk between objects: The comprehensive risk between objects can be obtained based on the basic risk and the distance risk. The calculation formula is as follows:

[0059] ;

[0060] As can be seen from the formula above, when the distance between objects is very close, the distance risk is high. Approaching 1, inter-object risk It will double; however, when the distance between objects is large, the distance risk... Approaching zero, inter-object risk This approach closely approximates the basic risk. This method effectively combines the inherent risks between objects with their relative distance, providing support for more accurate risk assessment.

[0061] ④ Construct a scene risk graph: treat each object in the scene as a node in the graph. The overall risk value between object pairs For border rights Construct a weighted undirected graph This forms the final "scenario risk map";

[0062] 2) Identify command type: Input user language commands and object information detected in the current scene, call the multimodal model to perform preliminary semantic analysis, and classify them into the following three categories according to the clarity of the command content:

[0063] a. Instructions that are too vague to be parsed: Language instructions contain pronouns that cannot be inferred (such as "it", "they") or ambiguous references (such as "this", "that"), making it impossible to determine the specific target;

[0064] b. Vague but clear instructions: Although there is semantic ambiguity, the intention can be clarified through contextual information or common sense reasoning. For example, "Give me the fruit" can be replaced with "Give me the apple".

[0065] c. Clear instructions: The language instructions clearly express the task objectives and the objects to be operated on;

[0066] 3) Instruction analysis and risk assessment:

[0067] ① Instructions that are too vague to be parsed: Do not speculate on the target object or behavior to avoid erroneous execution or potential security risks, and ensure that the task is carried out within a safe and controllable range;

[0068] ② Ambiguous but clear instructions: Use a multimodal model to generate several possible clear candidate instructions (e.g., the original instruction is "Please put the knife in a safe place"; five clear candidate instructions are generated: "Put the knife in the drawer", "Put the knife in the door crack", ..., "Put the knife in the trash can"). Combine the spaCy natural language processing tool to extract the key objects involved in each instruction and find their risk relationships in the scene risk map. Calculate the average risk score of each candidate instruction and select the one with the lowest risk ("Put the knife in the trash can") as the final execution instruction.

[0069] ③ Clear instructions: Retain the user's original instructions (such as putting the knife in the drawer) as the final execution instructions, without modifying them;

[0070] 4) Instruction rewriting and object identifier replacement:

[0071] The final execution instruction that has been judged or processed is rewritten and the object identifier is replaced, specifically including the following steps:

[0072] ③ Identify the optimal target object: When there are multiple objects of the same type or name in the scene (such as multiple trash cans in the scene), compare and analyze them in conjunction with the scene risk map, and select the object that is relevant to the task and has the lowest risk value as the final target to reduce the execution risk.

[0073] ④ Instruction text rewriting: Replace the natural language object names involved in the original instructions with the corresponding unique node numbers in the risk map, thereby realizing the conversion from natural language instructions to structured instructions.

[0074] ③ Output standard format: The output standard format adopts JSON format or other structured forms, which makes it easy for downstream modules to directly call and execute.

[0075] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the present invention. Although detailed descriptions have been provided with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments, and they should all be covered within the protection scope of the claims.

Claims

1. A method for recognizing embodied intelligent fuzzy language instructions based on scenario risk guidance, characterized in that, Includes the following steps: 1) Construct a scene risk map: Divide the risks between objects in the scene into two categories. One category is the basic risks brought about by the inherent characteristics or potential interactions of the objects, and the other category is the impact of the physical distance between the objects on these potential risks, i.e., distance risks. ① Basic Risk: The large language model infers and generates corresponding risk assessment values ​​based on the attributes, types, and relative relationships of objects. The basic risk is then divided into five levels according to these risk assessment values: a. Extremely low risk, basic risk assessment value (0, 0.2]: indicates that the risk is very low, and even if contact occurs, it is almost impossible to cause harm; b. Lower risk, basic risk assessment value A value of (0.2, 0.4) indicates a low risk; while there is a possibility of minor harm, it is generally acceptable, but appropriate caution is still required. c. Medium risk, basic risk assessment value A value of (0.4, 0.6) indicates that there is a certain risk; improper handling may result in damage to the item or minor personal injury. d. High risk, basic risk assessment value A value of (0.6, 0.8) indicates a high probability of injury, which may have serious consequences for personnel or equipment, requiring immediate preventive measures. e. Extremely high risk, basic risk assessment value (0.8, 1.0]: This indicates a serious risk; the coexistence of the two objects will almost certainly lead to catastrophic consequences, and immediate intervention is necessary. ② Distance risk: Assume the position coordinates of object A are... The position coordinates of object B are The formula for calculating the Euclidean distance d between them is as follows: ; When the Euclidean distance d between objects is greater than the safety threshold When the distance is zero, the risk is zero; however, when the Euclidean distance between objects is less than the safety threshold... In this case, the distance risk is normalized to the [0, 1] interval to unify the differences in distance scales across different scenarios and ensure that distance risks are comparable across different spatial scales. The distance risk formula is as follows: in, For distance risk; It is the attenuation coefficient, used to control the rate of attenuation; It is the base of the natural logarithm; ③ Calculation of comprehensive risk between objects: The comprehensive risk between objects can be obtained based on the basic risk and the distance risk. The calculation formula is as follows: ； ④ Construct a scene risk graph: treat each object in the scene as a node in the graph. The overall risk value between object pairs For border rights Construct a weighted undirected graph This forms the final "scenario risk map"; 2) Identify command type: Input user language commands and object information detected in the current scene, call the multimodal model to perform preliminary semantic analysis, and classify them into the following three categories according to the clarity of the command content: a. Instructions that are too vague to be parsed: The language instructions contain pronouns that cannot be inferred or ambiguous references, making it impossible to determine the specific target; b. Vague but clear instructions: Although there is semantic ambiguity, the intention can be clarified through contextual information or common sense reasoning; c. Clear instructions: The language instructions clearly express the task objectives and the objects to be operated on; 3) Instruction analysis and risk assessment: ① The instruction is too vague to be parsed: it does not infer the target object or behavior; ② Ambiguous but clear instructions: Use a multimodal model to generate several possible clear candidate instructions, combine spaCy natural language processing tools to extract the key objects involved in each instruction, find their risk relationships in the scene risk map, calculate the average risk score of each candidate instruction, and select the one with the lowest risk as the final execution instruction; ③ Clear instructions: Retain the user's original instructions as the final execution instructions without modification; 4) Instruction rewriting and object identifier replacement: The final execution instruction that has been judged or processed is rewritten and the object identifier is replaced, specifically including the following steps: ① Identify the optimal target object: When there are multiple objects of the same type or name in the scene, compare and analyze them in conjunction with the scene risk map, and select the object that is relevant to the task and has the lowest risk value as the final target to reduce the execution risk; ② Instruction text rewriting: Replace the natural language object names involved in the original instructions with the corresponding unique node numbers in the risk map, thereby realizing the conversion from natural language instructions to structured instructions; ③ Output standard format: The output standard format adopts a structured form, which makes it easy for downstream modules to directly call and execute.

2. The embodied intelligent fuzzy language instruction recognition method based on scenario risk guidance according to claim 1, characterized in that, The output standard format is JSON.

3. The embodied intelligent fuzzy language instruction recognition method based on scenario risk guidance according to claim 2, characterized in that, LLM specifically uses the GPT-4o model.

Images