Embodied intelligent safety evaluation method based on state change and event clustering

Abstract

The application provides a body intelligence safety evaluation method based on state change and event clustering, and belongs to the technical field of body intelligence safety evaluation. The method comprises the following steps: acquiring simulation data in a robot execution evaluation task, determining a preliminary attention object based on the respective pose data of each object in each simulation step; determining a neighboring object according to the distance between each preliminary attention object and other objects, thereby determining a target attention set; acquiring the state and linear motion parameter of each object in the target attention set, determining a structured state data record according to the respective pose data, the respective linear motion parameter and the respective state, thereby determining a plurality of structured change events, performing semantic conversion and inductive reasoning on the structured change events to obtain a change event description sequence; and evaluating the evaluation task according to the change event description sequence and a preset evaluation rule to obtain an evaluation score. The application can improve the fine granularity and interpretability of body intelligence safety evaluation.

Description

Technical Field

[0001] This application belongs to the field of embodied intelligence security evaluation technology, and more specifically, it relates to an embodied intelligence security evaluation method based on state change and event clustering. Background Technology

[0002] With the rapid development of embodied intelligence technology in tasks such as mobile operation and grasping / moving, physical interaction security issues are becoming increasingly prominent, severely restricting its large-scale deployment. Currently, mainstream embodied intelligence evaluation methods mainly include benchmark testing methods based on task completion rate, overall scoring methods based on large language models, and static security indicator detection methods oriented towards hardware interfaces.

[0003] However, the above methods are mostly task-oriented and lack systematic tracking and recording of intermediate process states such as object pose changes and contact relationship evolution during task execution, resulting in the inability to effectively reveal process safety hazards; the evaluation method based on large language model directly scores the overall output, and there is no traceable correspondence between the scoring process and the specific behavioral basis, resulting in insufficient interpretability of the evaluation results. Summary of the Invention

[0004] The purpose of this application is to provide an embodied intelligence security evaluation method based on state change and event clustering, so as to improve the fine-grainedness and interpretability of embodied intelligence security evaluation, and achieve the goal of automated and quantifiable evaluation of security behavior throughout the entire task execution process.

[0005] A first aspect of this application provides an embodied intelligence security evaluation method based on state changes and event clustering, including: The simulation data of the robot in the evaluation task is obtained. Based on the simulation data, the pose data of each object in each simulation step is obtained. Each object includes a first object and a second object. The first object and the second object are any two objects in the simulation environment. The simulation data is obtained through the simulation environment. The initial object of interest is determined based on the pose data of two adjacent simulation steps; For each initial object of interest, taking the initial object of interest as the center, determine each neighboring object based on the distance between the initial object of interest and other objects, and form a target interest set by the initial object of interest and each neighboring object; For each object in the target focus set, obtain the object's state and linear motion parameters. The linear motion parameters are determined by the pose data of the object in the simulation data for multiple consecutive simulation steps. The state includes self-check state and mutual check state. Based on each pose data, each linear motion parameter, and each state, determine the structured state data record; State change data is determined based on structured state records. Event-based clustering is performed on the state change data to form multiple structured change events. Semantic transformation is performed on each structured change event to generate its corresponding natural language description. Inductive reasoning is then performed on each natural language description to obtain a sequence of change event descriptions. State change data is used to record a real state change event. A structured change event refers to a structured data container formed by clustering multiple state change data with consecutive simulation step numbers according to time continuity. The evaluation task is evaluated based on the sequence of changing events and the preset evaluation rules to obtain an evaluation score.

[0006] A second aspect of this application provides an embodied intelligent security evaluation device based on state changes and event clustering, comprising: The data acquisition module acquires simulation data from the robot's evaluation tasks. Based on the simulation data, it acquires the pose data of each object in each simulation step. Each object includes a first object and a second object, which are any two objects in the simulation environment. The simulation data is obtained through the simulation environment. The initial object of interest determination module is used to determine the initial object of interest based on the pose data of two adjacent simulation steps. The target attention set determination module is used to determine each preliminary attention object, taking the preliminary attention object as the center, and determining each neighboring object based on the distance between the preliminary attention object and other objects, and forming a target attention set with the preliminary attention object and each neighboring object. The motion parameter acquisition module is used to acquire the state and linear motion parameters of each object in the target interest set. The linear motion parameters are determined by the pose data of the object in the simulation data of multiple consecutive simulation steps. The state includes self-check state and mutual check state. The state data recording module is used to determine the structured state data record based on each pose data, each linear motion parameter, and each state; The variable event description sequence determination module is used to determine state change data based on structured state records, perform event-based clustering on the state change data to form multiple structured change events, perform semantic transformation on each structured change event to generate its corresponding natural language description, and perform inductive reasoning on each natural language description to obtain the variable event description sequence. State change data is used to record a real state change event. Structured change event refers to a structured data container formed by clustering multiple state change data with consecutive simulation step number according to time continuity. The evaluation module is used to evaluate the evaluation task based on the sequence of changing events and preset evaluation rules, and obtain an evaluation score.

[0007] The beneficial effects of the embodied intelligent security evaluation method based on state change and event clustering provided in this application embodiment are as follows: This embodiment acquires the pose data of each object in the simulation scene and determines the target focus set based on each pose data, moving the evaluation anchor point from the task endpoint to the entire execution process, thus overcoming the deficiency of lacking systematic tracking of intermediate processes. Furthermore, this embodiment acquires the motion parameters and self-check / mutual-check status of the objects, transforming the raw perception data into structured state change records, then clustering them into change events according to temporal continuity and generating natural language descriptions, thereby refining scattered state changes into understandable plot units, i.e., structured change events. Based on this, this embodiment matches and comprehensively judges the event description sequence with preset evaluation rules, ensuring that each deduction can be accurately traced to a specific state change event, improving the traceability of the scoring basis and the consistency and objectivity of the evaluation results, and enhancing the scientific rigor and effectiveness of embodied intelligent safety evaluation. Attached Figure Description

[0008] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 A flowchart illustrating an embodiment of the embodied intelligence security evaluation method based on state change and event clustering provided in this application; Figure 2 This is a structural block diagram of an embodied intelligent security evaluation device based on state change and event clustering, provided as an embodiment of this application. Detailed Implementation

[0010] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0011] It should be noted that the large language model invocation involved in this application includes three independent stages: semantic inductive reasoning, termination condition verification, and evaluation task scoring. Each stage has different input content, reasoning tasks, and output goals, and they are executed independently without interference. During each invocation, preset prompts clearly define the task scope and output format of the current stage to ensure the accuracy and controllability of the reasoning results in each stage.

[0012] The large language model and text embedding model used in this application can both be pre-trained models commonly used in the field. Their specific network structures and training methods are existing technologies and are not part of the core improvements of this application. Those skilled in the art can use appropriate models to implement them according to actual needs, and this application will not elaborate on them. Among them, the large language model can be an open-source or commercial large language model such as the GPT series or the LLaMA series; the text embedding model can be a pre-trained model with semantic vectorization capabilities such as Sentence-BERT or text-embedding-ada-002.

[0013] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the embodied intelligence security assessment method based on state change and event clustering provided in this application. The method can be executed by an electronic device and may include: S101: Obtain simulation data from the robot's evaluation task, and based on the simulation data, obtain the pose data of each object in each simulation step.

[0014] In this embodiment, each object includes a first object and a second object, which are any two objects in the simulation environment. The simulation data is obtained through the simulation environment. The simulation step size refers to one iteration unit in which the simulation environment advances at fixed time intervals. Each simulation step size corresponds to a discrete time node, at which the simulation environment updates the physical state of all objects and outputs the corresponding scene data. The time interval between adjacent simulation steps can be determined by the sampling frequency of the simulation environment. Pose data includes position coordinate vectors and rotation angle vectors.

[0015] Before acquiring simulation data from the robot's evaluation task, this embodiment also includes configuring the simulation environment and running the simulation. Specifically, using a standardized evaluation configuration file as input, the simulation environment is configured to enable the robot to perform evaluation tasks, resource verification, and environmental parameter loading, thereby driving the simulation environment to complete the automated scheduling and execution of the evaluation task. The specific steps include: The standardized evaluation configuration file is read, and global settings are extracted from it. These global settings include key parameters such as evaluation task instructions, evaluation scenarios, evaluation dimensions, robot type, and the number of repeated evaluations. The task configuration is then expanded into independent task entries based on attributes such as evaluation dimensions, evaluation scenarios, and evaluation task instructions. Each task entry includes attributes such as its respective evaluation dimension, scene blank structure, object placement, and model information. The scene blank structure refers to the static, fixed basic environmental framework of the simulation scene, such as an empty room without furniture or items.

[0016] Before conducting simulation evaluation, all evaluation tasks are traversed to uniformly verify the integrity of the resource catalog corresponding to each task, preventing the evaluation process from being interrupted due to resource shortages; and a unified timestamp identifier is generated for all evaluation tasks in this batch. As a public index marker, it ensures data traceability and result reproducibility.

[0017] This embodiment first loads the configuration parameters required for the simulation environment, such as the coordinates and initial states of each object in the evaluation scenario. This embodiment uses the evaluation task instruction of "picking up a cup from the table and placing it in a designated area." When the robot executes this task instruction, it collides with the cup, causing the cup to tip over, roll off the table to the ground, and eventually roll next to the trash can. This embodiment uses this typical evaluation scenario of "a cup tipping over, rolling off the table to the ground, and eventually rolling next to the trash can" as an example to evaluate the "physical interaction safety during the robot's handling operation."

[0018] The evaluation scenario includes both robots and non-robots. Non-robot objects can include cups, tabletops, the ground, and trash cans. Based on this scenario, the first and second objects in this embodiment can be robots, cups, vases, trash cans, etc., in the simulation scene. A robot can refer to an embodied intelligent robot, which includes the robot body and the end effector of the robot. The position and angle of the robot body (the robot's chassis) and the end effector in this embodiment can be determined by the robot type. Viewpoint parameters (including pitch and yaw angle offsets) can be configured using offsets to adapt to the robot's execution viewpoint. In this evaluation scenario, the cup, as the configured task object, has initial coordinates of (1.2, 0.5, 0.8), which can be determined through random sampling. The trash can has fixed coordinates of (2.0, -1.0, 0.0). The ground, as the environmental plane, does not require coordinate determination. The tabletop, as the supporting surface, has a height determined by the scene's basic structure; for example, the tabletop's height is 0.75m. Based on the coordinates and attributes of each object, determine its initial state. For example, the cup is initially upright and its operable attribute is graspable; the trash can is initially empty and in a fixed position; the tabletop is stable. After loading all parameters, configure the simulation environment, drive the robot to perform the evaluation task, and obtain simulation data.

[0019] Based on the simulation data, and based on the scene corresponding to each simulation step in the simulation data, obtain the pose data of each object in the scene.

[0020] S102: Determine the initial object of interest based on the pose data of two adjacent simulation steps.

[0021] In one embodiment of this application, determining the initial object of interest based on the pose data of two adjacent simulation step lengths includes: For each object, obtain the pose data corresponding to the previous simulation step number and the next simulation step number in the simulation data. Perform a difference operation between the pose data corresponding to the next simulation step number and the pose data corresponding to the previous simulation step number to obtain the position offset and angle offset. If either the position offset or the angle offset exceeds the corresponding preset offset threshold, the object is identified as an object of initial interest.

[0022] In this embodiment, for each object, the pose data corresponding to the two adjacent simulation steps of that object in the simulation data are obtained. For example, the position coordinate vector corresponding to the t-th simulation step is... The position coordinate vector corresponding to the (t-1)th simulation step is The rotation angle vector corresponding to the t-th simulation step size is The rotation angle vector corresponding to the (t-1)th simulation step is .

[0023] The difference between the position coordinate vector corresponding to the next simulation step and the position coordinate vector corresponding to the previous simulation step is used to obtain the three-dimensional position offset of the object between two adjacent simulation steps. ,in, This represents the positional offset. The difference between the rotation angle vector corresponding to the next simulation step and the rotation angle vector corresponding to the previous simulation step is used to obtain the object's angular offset about its axis between two adjacent simulation steps. Specifically: ,in, This indicates the angular offset.

[0024] Compare the position offset and angle offset with their corresponding preset offset thresholds respectively. or ,in, This indicates the preset position offset threshold. The preset angle offset threshold indicates that the object is initially identified as an object of interest. In this embodiment, the preset position offset threshold can be 0.05m, and the preset angle offset threshold can be 10°.

[0025] In this embodiment, all objects corresponding to the first simulation step size in the simulation data are taken as the initial objects of interest.

[0026] For example, the cup is initially placed upright on the table, so the cup's position coordinates are... The angle of rotation of the cup is The robot's base position coordinates are (0.5, 0, 0). The end effector is performing a grasping action, and its position coordinates are... The rotation angle is When the robot executes the task instruction to "pick up the cup from the table and place it in the designated area," its end effector moves towards the cup. However, due to a path planning deviation, it collides with the cup wall during the approach, causing the cup to tip over. Starting from the t-th simulation step where the collision occurs, obtain the t-th simulation step and the t-th simulation step... Pose data for one simulated step size: For the cup, a collision causes its position coordinates to change from... Change to The cup's posture changed from upright to tilted, with an angular offset of approximately 35°. At this point, the cup's position offset was approximately 0.18m, exceeding the preset position offset threshold. The corresponding angular offset also exceeded the preset angular offset threshold. Based on this, the cup was identified as an object of initial concern.

[0027] For the actuator at the robot's end effector, its position coordinates are from... Change to Since the robotic arm did not retract significantly or make large adjustments after the collision, its posture remained basically unchanged. At this time, the positional offset of the actuator was about 0.017m, which did not exceed the preset positional offset threshold. The angular offset of the actuator also did not exceed the preset angular offset threshold. Based on this, the actuator at the end of the robot was not identified as a preliminary object of concern.

[0028] S103: For each initial object of interest, take the initial object of interest as the center, determine each neighboring object based on the distance between the initial object of interest and other objects, and form a target interest set with the initial object of interest and each neighboring object.

[0029] In one embodiment of this application, determining neighboring objects based on the distance between the initial object of interest and other objects includes: Calculate the minimum spatial distance between the bounding box of the initial object of interest and the bounding boxes of all other objects in the scene; Other objects whose minimum spatial distance is less than or equal to a preset distance threshold are considered as neighboring objects of the initial object of interest.

[0030] In this embodiment, a bounding box refers to the smallest axis-aligned cube or cuboid enclosing space established for each object in the scene, used to characterize the area occupied by the object in three-dimensional space. Its boundary can be determined by the maximum and minimum values ​​of the object in each axis. The minimum spatial distance refers to the shortest Euclidean distance between the surfaces of the bounding boxes of two objects. When the two bounding boxes intersect or penetrate each other, the minimum spatial distance is zero. A preset distance threshold is used to determine whether two objects have a spatial proximity relationship in the sense of safety assessment.

[0031] For each initial object of interest, obtain the spatial extent information of the bounding box of that object in the simulation scene, as well as the spatial extent information of the bounding boxes of other objects in the scene corresponding to that simulation step.

[0032] Using the initial object of interest as the center, calculate the minimum spatial distance between the bounding box of the initial object of interest and the bounding boxes of all other objects in the scene. .

[0033] For each other object, find the minimum spatial distance corresponding to that other object. and preset distance threshold If a comparison is made, If so, then the other object is considered a neighboring object of the initial object of interest; if If so, then the other object is determined to be a non-nearby object.

[0034] In this embodiment, the initial object of interest and all its neighboring objects are grouped into the target interest set.

[0035] To further enhance the comprehensiveness of the evaluation task, this embodiment, after determining the target focus set, synchronously triggers an expanded search of the robot body and its surrounding objects based on changes in the end effector. This is because when the robot collides with the cup wall, the end effector may not be initially identified as an object of interest due to its gentle movement. In subsequent simulation steps, if the distance between the end effector and other objects in the scene is large, these objects may not be included in the focus set, potentially missing potential interaction risks.

[0036] As can be seen from the above, this embodiment is based on the nearest neighbor expansion mechanism of the minimum spatial distance of the bounding box. With spatial distance as the criterion, it achieves automatic coverage of all potential risk-related objects around the object of initial concern without relying on predefined semantic relationships or complex interaction models. This makes up for the missed detection problem that may be caused by relying solely on pose changes for screening, and improves the completeness of process data collection and the reliability of evaluation results in the embodied safety evaluation process.

[0037] S104: For each object in the target focus set, obtain the object's state and linear motion parameters.

[0038] In this embodiment, the linear motion parameters are determined by the pose data of the object over multiple consecutive simulation steps in the simulation data. The linear motion parameters include linear velocity, linear acceleration, and linear jerk. The states include self-check state and mutual check state.

[0039] In one embodiment of this application, for each object in the target interest set, obtaining the state of that object includes: For each object in the target focus set, obtain the object's own attributes and use these attributes as the object's self-check status. The object's self-check status includes restricted area entry status, attitude data, velocity data, opening and closing status, and temperature. For any pair of objects in the target focus set, obtain the relative relationship between the two objects. The mutual check status between the two objects is determined based on their relative relationship; the mutual check status includes contact status, overlapping status, proximity status, and support status.

[0040] In this embodiment, the self-check status refers to the inherent attributes or state information of a single object, independent of its relative relationship with other objects. This includes: restricted area entry status (whether the object is within a preset virtual restricted area), posture data (the object's orientation or configuration in space; for robots, this also includes the angles of each joint), speed data (the object's linear velocity; exceeding a preset speed threshold indicates overspeeding; the preset speed threshold can be set according to scenario safety specifications, for example, a robot linear velocity threshold of 0.5 m / s), opening / closing status (the degree of opening or closing of the object; for robots, this refers to the opening degree of the end effector, represented by continuous values ​​between 0 and 1, where 0 represents complete closure and 1 represents complete opening), and temperature (the operating temperature of the robot's joints or motors, in degrees Celsius). The determination of restricted area entry status, posture data, speed data, opening / closing status, and robot speed in the above self-check status can be implemented according to a pre-designed algorithm; joint opening / closing, temperature, and other state quantities can be directly read by calling the open-source application programming interface of the simulation environment. The method for determining the entry status of a restricted area can be as follows: Obtain the range information of the object's axis-aligned bounding box (AABB) in 3D space; perform overlap detection between the bounding box and a preset restricted area range (e.g., a 3D axis-aligned bounding box); if the two intersect in all three axes, the object is determined to have entered the restricted area. The restricted area range can be pre-set by the tester according to the safety specifications of the evaluation scenario. The method for determining speeding in the velocity data can be as follows: Real-time detection of the object's jerk value; when the jerk exceeds a preset threshold, the current velocity value is recorded as the velocity of the point of interest (POI), with each POI velocity corresponding to a potential speeding risk moment; in the subsequent rule evaluation stage, the large language model performs semantic verification on all POI velocities according to the preset velocity threshold for the current task, and determines the velocity exceeding the task threshold as speeding and deducts points accordingly. By combining POI extraction with semantic verification, the computational redundancy of comparing all velocity data is avoided, improving system efficiency while ensuring the accuracy of safety evaluation.

[0041] Mutual status refers to the state information arising from the spatial relative relationship between any two objects in the target focus set, including: contact state (the distance between the surfaces of the two objects is zero or penetration occurs), overlap state (the bounding boxes of the two objects intersect in space, i.e., occupy the same spatial area), proximity state (the minimum spatial distance between the surfaces of the two objects is less than a preset proximity threshold, such as less than 0.05 meters), and support state (one object is located above another object and the surfaces of the two objects are in contact, forming a support relationship). Among these, the contact state can be determined by a combination of the contact relationship and velocity between the objects (for example, when two objects are in contact and the relative velocity is not zero, it is determined that there is a risk of collision), while other mutual statuses can be directly read by the open-source application programming interface of the simulation environment.

[0042] In this embodiment, for each object, the restricted area entry status, attitude data, opening and closing status, temperature and speed data are used as the object's self-check status.

[0043] For all objects in the target focus set, they are paired up to form several pairing combinations. For each pairing combination, the spatial position and bounding box information of each of the two objects are obtained. Based on the spatial position and bounding box information, the spatial relative relationship between the two objects is calculated, and the mutual lookup state of the pairing combination is determined according to the calculation results. Specifically, if the surface distance between the two objects is detected to be zero or negative, it is determined to be a contact state; if the bounding boxes of the two objects are detected to overlap, it is determined to be an overlapping state; if the minimum spatial distance between the surfaces of the two objects is detected to be greater than zero but less than a preset proximity threshold (e.g., 0.05 meters), it is determined to be a proximity state; if one object is detected to be above another object and the surfaces of the two objects are in contact, it is determined to be a support state.

[0044] If an object does not support querying a specific state value due to its own data type or attributes (for example, rigid objects such as cups and tabletops do not have temperature attributes), the abnormal state combination will be automatically recorded and the invalid detection item will be skipped in subsequent collections to avoid performance loss caused by invalid calls.

[0045] In this embodiment, the linear motion parameters are determined in the following way: For each object in the target focus set, obtain the position sequence of that object within multiple consecutive simulation steps; Determine the linear velocity based on the first derivative of the position sequence with respect to time; Determine linear acceleration based on the first derivative of linear velocity with respect to time; The linear jerk is determined by the first derivative of the linear acceleration with respect to time.

[0046] In this embodiment, if the object is a robot object, in addition to obtaining the linear motion parameters of the object, it is also necessary to obtain the angular motion parameters of the robot object. The angular motion parameters include angular velocity, angular acceleration, and angular jerk. The angular motion parameters are determined in the following way: For each robot object in the target focus set, obtain the rotation angle sequence of that robot object within multiple consecutive simulation steps; Angular velocity is determined based on the first derivative of the rotation angle sequence with respect to time. Angular acceleration is determined by the first derivative of angular velocity with respect to time. Angular jerk is determined by the first derivative of angular acceleration with respect to time.

[0047] In this embodiment, the linear velocity of each object in the target interest set is calculated based on the position coordinates of multiple consecutive simulation steps. Linear acceleration and linear accelerometer If the target focus set includes robot objects, it is also necessary to obtain the angular velocity of those robot objects. angular acceleration and angular acceleration .

[0048] Specifically, the linear motion parameters can be calculated as follows: obtain the position sequence of the object within multiple consecutive simulation steps, take the first derivative of the position sequence with respect to time to obtain the linear velocity, take the first derivative of the linear velocity to obtain the linear acceleration, and take the first derivative of the linear acceleration to obtain the linear jerk. Since the sampling rate of the simulation data is fixed and the time interval between adjacent simulation steps is constant, the finite difference method can be used to approximate the above derivatives. The calculation method for angular motion parameters is similar to that for linear motion parameters: obtain the rotation angle sequence of the object within multiple consecutive simulation steps, take the first derivative of the rotation angle sequence with respect to time to obtain the angular velocity, take the first derivative of the angular velocity to obtain the angular acceleration, and take the first derivative of the angular acceleration to obtain the angular jerk.

[0049] In this embodiment, linear jerk and angular jerk are third-order kinematic quantities, which are highly sensitive to perceived noise. Smoothing processing can be used in the calculation to suppress the impact of noise on the accuracy of higher-order parameter calculations, thus ensuring the reliability of the kinematic parameters. The linear and angular motion parameters obtained in this embodiment are used to support subsequent determination of overspeed states and collision detection for each object in the target interest set.

[0050] S105: Based on each pose data, each linear motion parameter, and each state, determine the structured state data record.

[0051] In this embodiment, structured state data record refers to a standardized data unit formed by summarizing and encapsulating various types of perception data of all objects of interest within the same simulation step according to a preset data structure template. Each record corresponds to the complete state information of an object in one simulation step.

[0052] In this embodiment, after obtaining the pose data, linear motion parameters, and status (including self-check status and mutual check status) of each object in the target interest set, the above data are integrated into a unified structured status data record.

[0053] Specifically, for the current simulation step size, traverse each object in the target interest set and perform the following steps: For each object in the target focus set, based on the obtained pose data, linear motion parameters, and the object's self-check status and the object's mutual check status with other objects in the set, a structured state data record of the object at the current simulation step is formed.

[0054] If the object is a robot object, the obtained angular motion parameters need to be included in the structured state data record of the robot object based on the structured data to ensure that the rotational kinematics information of the robot object is completely recorded; if the current object is a non-robot object, there is no need to include the angular motion parameters.

[0055] As can be seen from the above, this embodiment aggregates the structured state data records formed by all objects in the target interest set within the current simulation step according to the object identifier, forming a set of structured state data records for the current simulation step, which can be called by subsequent state change detection and event-based clustering steps.

[0056] S106: Determine state change data based on structured state records, perform event-based clustering on the state change data to form multiple structured change events, perform semantic transformation on each structured change event to generate its corresponding natural language description, and perform inductive reasoning on each natural language description to obtain a sequence of change event descriptions.

[0057] In this embodiment, state change data is used to record a real state change event; a structured change event refers to a structured data container formed by clustering multiple state change data with consecutive simulation step size numbers according to temporal continuity. The change event description sequence refers to a serialized data structure formed by arranging high-level natural language event descriptions generated after inference and summarization of each structured change event through a large language model, according to the chronological order of the simulation step size numbers of each event. A preset difference threshold is used to determine the upper limit of the difference in simulation step size numbers between adjacent state changes and whether they belong to the same change event; in this embodiment, the preset difference threshold is set to 20 steps.

[0058] In one embodiment of this application, determining state change data based on structured state records includes: For each object, the self-check status of the object is expanded into a first key-value pair, and the mutual check status is expanded into a second key-value pair. Each key-value pair includes a status identifier and a status value. The first key-value pair and the second key-value pair are compared item by item with the historical key-value pairs already stored in the corresponding objects in the global state table to filter out the key-value pairs whose state values ​​have changed. For each key-value pair that has changed, if the key-value pair belongs to a preset security sensitivity type, then the key-value pair is determined to be a key-value pair that has actually changed; where security sensitivity types include temperature data, attitude data, and velocity data; If the key-value pair does not belong to the preset security-sensitive type, a sliding window debouncing mechanism is used to determine whether the key-value pair is a real change key-value pair; Update the state value corresponding to the object in the global state table based on the key-value pairs that have actually changed. Generate the state change data corresponding to the object based on the updated global state table; The method for determining whether a key-value pair is a truly changed key-value pair using a sliding window debouncing mechanism and preset judgment conditions includes: Within a sliding window of length N simulation steps, count the number of simulation steps in which the state value of the key-value pair continuously appears. If the number of simulation steps reaches the preset step size threshold, the key-value pair is determined to be a key-value pair that has actually changed; otherwise, the key-value pair is determined to have not changed.

[0059] In this embodiment, key-value pairs refer to the key-value correspondence formed by using the state name as the key and the specific value of the state as the value, used to uniformly represent various types of state information. The global state table refers to the storage structure that maintains the latest confirmed state values ​​of all objects. The sliding window debouncing mechanism refers to a method that statistically confirms the observation values ​​of the same state item within multiple consecutive simulation steps, used to filter out false state changes caused by perceived noise or instantaneous fluctuations. In this embodiment, the length of the sliding window can be set to 5 steps, and the preset step threshold can be set to 4 steps. Safety-sensitive types refer to state types that require immediate response and involve physical safety risks, including temperature data, attitude data, and velocity data.

[0060] In this embodiment, for each object in the structured state data record, the self-check state of the object is expanded into a first key-value pair, and the mutual check state is expanded into a second key-value pair. Specifically, for the self-check state, "object identifier + state name" is used as the key, and the specific value of the state is used as the value. In this embodiment, the self-check state can be represented as "self|object name|state name". For the mutual check state, "main object identifier + reference object identifier + state name" is used as the key, and the specific value of the pair in the state item is used as the value. In this embodiment, the mutual check state can be represented as "mutual|main|reference|state name". All key-value pairs are summarized to obtain the complete set of state key-value pairs for the current simulation step size.

[0061] The entire set of state key-value pairs is compared item by item with the historical key-value pairs stored in the global state table. Key-value pairs with the same key but different values ​​are selected as candidate changes.

[0062] For each candidate change, its state type is determined to be a preset security-sensitive type. If it is, the change is directly identified as a real key-value pair, and subsequent debouncing steps are skipped. If the state type of the candidate change does not belong to the preset security-sensitive type, a sliding window debouncing mechanism is used for confirmation. Specifically, the current simulation step size is taken as t, and a sliding window of length N (i.e., from the t-th step) is used. Within N+1 simulation steps to the t-th simulation step, count the new value of the state term. The simulated step size is used to determine whether the change is a real change based on a preset judgment condition. This preset judgment condition is... , N Indicates the length of the sliding window. M This represents the preset step size threshold within the sliding window. Indicates the first Each simulation step corresponds to the acquired value of the state item. This indicates a new status value that needs to be confirmed. This indicates a change confirmation flag. If the change confirmation flag... If the value of is 1, the change is determined to be a real change; otherwise, the change is determined to be a transient fluctuation or noise and is not confirmed.

[0063] Based on all key-value pairs determined to be real changes, update the current value of the corresponding object in the global state table for that state entry. Based on the updated global state table, generate corresponding state change data for each real change. Each state change data entry must include at least the main object identifier, the reference object identifier (if the state is a mutual lookup state, fill in the corresponding reference object; if it is a self-lookup state, this field is empty), the state name, the old value, and the new value.

[0064] To further improve the accuracy of state change data and avoid the same physical interaction event being recorded repeatedly due to the interchange of subject and object perspectives, this embodiment performs a subject-object interchange deduplication operation on mutual check states with subject-object symmetry (such as contact state, collision state, etc.).

[0065] In one embodiment of this application, event-based clustering is performed on state change data to form multiple structured change events, including: The state change data sequence is determined based on each state change data and its corresponding simulation step size number. Starting with the first state change data in the state change data sequence, iterate through each state change data in the state change data sequence in turn. For the current state change data, calculate the difference between its simulation step size number and the simulation step size number of the previous state change data. If the difference in sequence number is less than or equal to the preset difference threshold, then the currently traversed state change data will be included in the current change event. If the difference in sequence number is greater than the preset difference threshold, the current change event is terminated, and a new change event is created using the current state change data as the starting change data. Structured change events are generated based on each change event.

[0066] In this embodiment, all state change data are arranged in ascending order of simulation step size number to form a state change data sequence. Starting with the first state change data in the state change data sequence, a new first change event is created, the first data is assigned to this change event, and the starting step size number of this change event is recorded.

[0067] The simulation step size sequence is iterated through sequentially. For each state change data entry, its simulation step size number and the simulation step size number of the previous state change data are obtained, and the difference between the two numbers is calculated. This difference is compared with a preset difference threshold. If the difference is less than or equal to the preset difference threshold, it is determined that the current state change data and the previous state change data are continuous in time and belong to the continuation of the same change event. The current state change data is then included in the current change event, and the termination step size number of the event is updated to the current step size number.

[0068] If the difference in sequence number is greater than the preset difference threshold, it is determined that the previous change event has terminated, and the current state change data belongs to the beginning of a new event. At this time, the current change event is closed and saved, and then a new change event is created starting from the current state change data. The current change event is then included in the new change event and its starting step number is recorded.

[0069] Repeat the above steps until all state change data has been traversed, forming multiple structured change events. Each structured change event includes all state change data belonging to the same change event and its time range information.

[0070] In one embodiment of this application, semantic transformation is performed on each structured change event to generate its corresponding natural language description, and inductive reasoning is performed on each natural language description to obtain a sequence of change event descriptions, including: For each structured change event, the state change data contained in the structured change event are converted into their respective natural language descriptions according to the preset logical relationship. Based on the natural language descriptions, determine the event text, and then determine the sequence of changing event descriptions based on the event texts.

[0071] In this embodiment, the preset logical relationship refers to the fixed information extraction order used to convert each state change data into a natural language description, including the main object, state name, reference object (if the state is a mutual query state), old value, and new value.

[0072] For each structured change event, obtain all state change data contained in that structured change event. Each state change data entry includes the main object identifier, state name, reference object identifier (if it is a cross-reference state), old value, and new value. Extract the corresponding field information from each state change data entry sequentially according to the preset logical relationship, that is, extract the main object, state name, reference object (if any), old value, and new value in sequence.

[0073] Based on the state type corresponding to the state change data, a semantic template matching the state type is selected from a pre-built differentiated semantic template library. The extracted field information is then filled into the corresponding positions of the template to generate a natural language description of the state change data. In this embodiment, the differentiated semantic template library refers to a set of pre-built natural language expression templates with different sentence structures for different state types. For example, if a state change data is a mutual query state, its main object is the robot end effector, the reference object is the cup, the state name is contact state, the old value is "not in contact", and the new value is "in contact", then the corresponding natural language description "the contact state between the robot end effector and the cup has changed from not in contact to in contact" is generated.

[0074] In this embodiment, during the conversion process, repetitive or redundant descriptions are filtered out. For example, if the same state is repeatedly generated with the same semantics in multiple consecutive simulation steps, only the first occurrence and the final change result are retained.

[0075] For all state change data within the same structured change event, the natural language descriptions are arranged sequentially according to their respective simulation step size numbers from smallest to largest to form the complete event text of the change event, ensuring that the temporal logic between the state changes within the event is preserved in the text.

[0076] This embodiment uses the complete event text of each change event as input to the large language model. Preset prompts guide the model to perform overall reasoning and summarization of all state change descriptions within the event text. This allows it to transcend the isolated perspective of a single state change, automatically correlate and causally infer multiple state changes within an event, and uncover the causal relationships and evolutionary chains between them. For example, when a change event contains multiple state change descriptions such as "the robot's end effector contacts the cup," "the cup's posture changes from upright to tilted," and "the cup's position continues to descend," the large language model can correlate these state changes and perform causal reasoning, summarizing and generating a high-level causal event description: "The cup tipped over due to the robot's collision and rolled from the table to the ground." Furthermore, the large language model automatically filters residual jitter noise in the event text during the summarization process, ensuring that the output event description focuses on the substantial physical change process.

[0077] In this embodiment, the high-level natural language event descriptions generated by the reasoning and induction of each structured change event through the large language model are arranged in the order of the starting step size number of each event to form the final change event description sequence.

[0078] After forming the final sequence of change events, this embodiment will combine preset task success conditions and failure termination conditions to re-invoke the large language model to verify the newly added events. Specifically, the latest change event in the change event description sequence, along with the preset task success conditions and failure termination conditions, are input into the large language model. The model then comprehensively determines whether the current task meets the success conditions or triggers the failure termination conditions. If any termination condition is met, a termination signal is triggered, ending the data acquisition process of the current simulation task to avoid the continuous recording and processing of subsequent invalid states. If no termination condition is met, the simulation task continues to execute and proceeds to the next step. In addition, this embodiment is also equipped with a maximum simulation step timeout protection mechanism. When the cumulative number of simulation steps exceeds a preset limit, the simulation is automatically and forcibly terminated as a backup guarantee in case of large language model verification failure or abnormality, ensuring that the simulation process will not continue indefinitely due to unexpected circumstances.

[0079] As can be seen from the above, this embodiment, by filtering out actual changes and generating standardized state change data, completes the noise reduction and extraction from raw perceived data to effective change signals. Based on the continuity of step-size sequence numbers, it aggregates state changes scattered across different step sizes into independent change events according to temporal proximity, raising the data organization granularity to the event level and solving the problems of information fragmentation and difficulty in directly supporting high-level semantic understanding. Furthermore, this embodiment extracts key fields from the structured change data within each change event according to preset logical relationships and transforms them into natural language descriptions using differentiated semantic templates. This transforms machine-readable structured data into natural language expressions understandable by humans and large language models, achieving an effective conversion from structured data to event-level natural language description sequences, and improving the interpretability and automation level of embodied intelligence security assessment.

[0080] S107: Evaluate the evaluation task based on the sequence of changing events and the preset evaluation rules, and obtain the evaluation score.

[0081] In this embodiment, various evaluation rules are pre-organized and archived in natural language. Each evaluation rule includes the deduction object and the benchmark deduction value. and floating range Furthermore, the text length of each evaluation rule is controlled to be within 150 characters. This embodiment utilizes a text embedding model to convert each rule text into a corresponding vector representation and establishes an index for all rule vectors, forming a rule vector library to support subsequent efficient semantic similarity retrieval. The text embedding model refers to a neural network model capable of mapping natural language text into fixed-dimensional dense vectors. This model is trained using large-scale text corpora for comparative learning, ensuring that semantically similar texts are close in the vector space, thus supporting semantic retrieval based on vector similarity.

[0082] For each change event in the change event description sequence, a large language model is used to extract the core elements of the event. These core elements include the event type, involved parties, consequences and risk levels, and task relevance. Based on the extracted core elements, a retrieval text vector for the event is generated. Then, the cosine similarity between this retrieval text vector and each evaluation rule vector in the rule vector library is calculated. The cosine similarities are sorted from high to low, and the Top N evaluation rules with the highest similarity are selected as candidate rules for the change event.

[0083] The candidate rules and a complete description of the change event are input into the large language model. After comprehensive evaluation, the large language model selects the best-matching rule from the candidate rules and deducts a benchmark score based on the rule. and floating range The actual deduction value for the change event is output based on its specific severity. The system iterates through all change events within the task cycle, obtaining a corresponding deduction value for each event. By summing all deduction values, the final safety score for the task is calculated, improving the recall accuracy of rule matching and providing a reliable basis for the output deduction values, thus enhancing the consistency and reproducibility of the evaluation results. For example, a change event described as "the robot's end effector collides with a cup, causing the cup's posture to change from upright to tilted" might be considered. Candidate rules for recall might include "the robot collision causes the object to tip over, with a base deduction of 5 points and a fluctuation of ±2 points." After comprehensive evaluation using a large language model, the system determines that the change event matches this candidate rule and that the event's severity is high, resulting in a final output deduction value of 7 points.

[0084] In this embodiment, each assessment task is repeated a set number of times. Repeat the evaluation process to obtain multiple scores, and use the arithmetic mean of the multiple scores as the final average score for the evaluation task. After all assessment tasks have been completed, the average scores for each task belonging to the same assessment dimension will be summarized and statistically analyzed to calculate the average score for each dimension. It also counts the number of tasks under each dimension, thereby generating a complete evaluation report that includes details of deductions for each task and the final score, the average score of multiple repeated evaluations, and the average score and task count statistics summarized by evaluation dimension.

[0085] As can be seen from the above, this embodiment, by pre-constructing a vectorized natural language rule base and combining it with a text embedding model, achieves semantic similarity retrieval between rules and changing events. It efficiently matches event descriptions with the massive number of rules in the rule base, overcoming the limitations of poor semantic generalization ability in traditional keyword matching methods and improving the accuracy and coverage of rule recall. Furthermore, through a comprehensive evaluation of candidate rules and complete event descriptions using a large language model, the determination of deduction values ​​is based on a dual consideration of rule baseline values ​​and the actual severity of the event. This ensures that each deduction has a clear behavioral basis and a reasonable quantitative scale, overcoming the lack of traceability in traditional overall scoring methods.

[0086] This embodiment forms a closed-loop "four-level pipeline" from state acquisition, change description, event reasoning to rule retrieval and quantitative scoring. It requires no human intervention, and the evaluation process is objective and consistent, effectively overcoming the inherent limitations of traditional evaluation methods, such as strong subjectivity, poor generalization, and low efficiency. Furthermore, this embodiment can perform batch and standardized security evaluations on massive simulation tasks, providing a scalable engineering infrastructure for the systematic security verification and iterative optimization of embodied intelligent products.

[0087] Corresponding to the embodied intelligence security evaluation method based on state change and event clustering in the above embodiments, Figure 2This is a structural block diagram of an embodied intelligent security evaluation device based on state change and event clustering, provided as an embodiment of this application. For ease of explanation, only the parts relevant to the embodiment of this application are shown. References Figure 2 The embodied intelligent safety assessment device 20 based on state change and event clustering includes: a data acquisition module 21, a preliminary object of interest determination module 22, a target interest set determination module 23, a motion parameter acquisition module 24, a state data recording module 25, a change event description sequence determination module 26, and an assessment module 27.

[0088] Among them, the data acquisition module 21 acquires the simulation data in the robot's evaluation task, and based on the simulation data, acquires the pose data of each object in each simulation step; each object includes a first object and a second object, the first object and the second object are any two objects in the simulation environment, and the simulation data is obtained through the simulation environment; The initial object of interest determination module 22 is used to determine the initial object of interest based on the pose data of two adjacent simulation steps; The target attention set determination module 23 is used to determine each neighboring object based on the distance between the initial attention object and other objects, with the initial attention object as the center, and form a target attention set with the initial attention object and each neighboring object. The motion parameter acquisition module 24 is used to acquire the state and linear motion parameters of each object in the target interest set. The linear motion parameters are determined by the pose data of the object in the simulation data of multiple consecutive simulation steps. The state includes self-check state and mutual check state. The state data recording module 25 is used to determine the structured state data record based on each pose data, each linear motion parameter and each state; The variable event description sequence determination module 26 is used to determine state change data based on structured state records, perform event-based clustering on the state change data to form multiple structured change events, perform semantic transformation on each structured change event to generate its corresponding natural language description, and perform inductive reasoning on each natural language description to obtain a variable event description sequence; the state change data is used to record a real state change event; the structured change event refers to a structured data container formed by clustering multiple state change data with consecutive simulation step number according to time continuity. The evaluation module 27 is used to evaluate the evaluation task based on the sequence of changing events and preset evaluation rules, and obtain an evaluation score.

[0089] In one embodiment of this application, when determining state change data based on structured state records, the change event description sequence determination module 26 is specifically used for: For each object, the self-check status of the object is expanded into a first key-value pair, and the mutual check status is expanded into a second key-value pair. Each key-value pair includes a status identifier and a status value. The first key-value pair and the second key-value pair are compared item by item with the historical key-value pairs already stored in the corresponding objects in the global state table to filter out the key-value pairs whose state values ​​have changed. For each key-value pair that has changed, if the key-value pair belongs to a preset security sensitivity type, then the key-value pair is determined to be a key-value pair that has actually changed; where security sensitivity types include temperature data, attitude data, and velocity data; If the key-value pair does not belong to the preset security-sensitive type, a sliding window debouncing mechanism is used to determine whether the key-value pair is a real change key-value pair; Update the state value corresponding to the object in the global state table based on the key-value pairs that have actually changed. Generate the state change data corresponding to the object based on the updated global state table.

[0090] In one embodiment of this application, when using a sliding window debouncing mechanism to determine whether a key-value pair is a truly changed key-value pair, the change event description sequence determination module 26 is specifically used for: Within a sliding window of length N simulation steps, count the number of simulation steps in which the state value of the key-value pair continuously appears. If the number of simulation steps reaches the preset step size threshold, the key-value pair is determined to be a key-value pair that has actually changed; otherwise, the key-value pair is determined to have not changed.

[0091] In one embodiment of this application, when performing event-based clustering on state change data to form multiple structured change events, the change event description sequence determination module 26 is specifically used for: The state change data sequence is determined based on each state change data and its corresponding simulation step size number. Starting with the first state change data in the state change data sequence, iterate through each state change data in the state change data sequence in turn. For the current state change data, calculate the difference between its simulation step size number and the simulation step size number of the previous state change data. If the difference in sequence number is less than or equal to the preset difference threshold, then the currently traversed state change data will be included in the current change event. If the difference in sequence number is greater than the preset difference threshold, the current change event is terminated, and a new change event is created using the current state change data as the starting change data. Structured change events are generated based on each change event.

[0092] In one embodiment of this application, when semantically transforming each structured change event to generate its corresponding natural language description, and performing inductive reasoning on each natural language description to obtain a change event description sequence, the change event description sequence determination module 26 is specifically used for: For each structured change event, the state change data contained in the structured change event are converted into their respective natural language descriptions according to the preset logical relationship. The preset logical relationship includes the main object, state name, reference object, old value and new value. Based on the natural language descriptions, determine the event text, and then determine the sequence of changing event descriptions based on the event texts.

[0093] In one embodiment of this application, the pose data includes coordinates and rotation angles. When determining the initial object of interest based on the pose data of two adjacent simulation step sizes, the initial object of interest determination module 22 is specifically used for: For each object, obtain the pose data corresponding to the previous simulation step number and the next simulation step number in the simulation data. Perform a difference operation between the pose data corresponding to the next simulation step number and the pose data corresponding to the previous simulation step number to obtain the position offset and angle offset. If either the position offset or the angle offset exceeds the corresponding preset offset threshold, the object is identified as an object of initial interest.

[0094] In one embodiment of this application, when determining neighboring objects based on the distance between the initial object of interest and other objects, the target interest set determination module 23 is specifically used for: Calculate the minimum spatial distance between the bounding box of the initial object of interest and the bounding boxes of all other objects in the scene; Other objects whose minimum spatial distance is less than or equal to a preset distance threshold are considered as neighboring objects of the initial object of interest.

[0095] In one embodiment of this application, when acquiring the state of each object in the target interest set, the motion parameter acquisition module 24 is specifically used for: For each object in the target focus set, obtain the object's own attributes and use these attributes as the object's self-check status. The object's self-check status includes restricted area entry status, attitude data, velocity data, opening and closing status, and temperature. For any pair of objects in the target focus set, obtain the relative relationship between the two objects. The mutual check status between the two objects is determined based on their relative relationship; the mutual check status includes contact status, overlapping status, proximity status, and support status.

[0096] In one embodiment of this application, the linear motion parameters include linear velocity, linear acceleration, and linear jerk. When determining the linear motion parameters, the motion parameter acquisition module 24 is specifically used for: For each object in the target focus set, obtain the position sequence of that object within multiple consecutive simulation steps; Determine the linear velocity based on the first derivative of the position sequence with respect to time; Determine linear acceleration based on the first derivative of linear velocity with respect to time; The linear jerk is determined by the first derivative of the linear acceleration with respect to time.

[0097] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces or units, or they may be electrical, mechanical, or other forms of connection.

[0098] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0099] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0100] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for evaluating embodied intelligent security based on state changes and event clustering, characterized in that, include: Acquire simulation data from the robot's performance evaluation task, and based on the simulation data, acquire the pose data of each object in each simulation step. Each object includes a first object and a second object, wherein the first object and the second object are any two objects in the simulation environment, and the simulation data is obtained through the simulation environment; The initial object of interest is determined based on the pose data of two adjacent simulation steps; For each initial object of interest, taking the initial object of interest as the center, determine each neighboring object based on the distance between the initial object of interest and other objects, and form a target interest set by the initial object of interest and each of the neighboring objects. For each object in the target focus set, the state and linear motion parameters of the object are obtained. The linear motion parameters are determined by the pose data of the object in the simulation data for multiple consecutive simulation steps. The state includes self-check state and mutual check state. Based on the pose data, linear motion parameters, and states, a structured state data record is determined. State change data is determined based on the structured state records. Event-based clustering is performed on the state change data to form multiple structured change events. Semantic transformation is performed on each structured change event to generate its corresponding natural language description. Inductive reasoning is then performed on each natural language description to obtain a sequence of change event descriptions. The state change data is used to record a real state change event. The structured change event refers to a structured data container formed by clustering multiple consecutive state change data with continuous simulation step numbers according to temporal continuity. The evaluation task is evaluated based on the sequence of change events and preset evaluation rules to obtain an evaluation score.

2. The method as described in claim 1, characterized in that, The step of determining the state change data based on the structured state record includes: For each object, the self-check status of the object is expanded into a first key-value pair, and the mutual check status is expanded into a second key-value pair, wherein each key-value pair includes a status identifier and a status value; The first key-value pair and the second key-value pair are compared item by item with the historical key-value pairs already stored in the corresponding objects in the global state table to filter out the key-value pairs in which the state value has changed. For each key-value pair that has changed, if the key-value pair belongs to a preset security sensitivity type, then the key-value pair is determined to be a key-value pair that has actually changed; wherein, the security sensitivity type includes temperature data, attitude data, and speed data; If the key-value pair does not belong to the preset security-sensitive type, a sliding window debouncing mechanism is used to determine whether the key-value pair is a real change key-value pair; Update the state value corresponding to the object in the global state table based on the actual changes in the key-value pairs; Generate the state change data corresponding to the object based on the updated global state table.

3. The method as described in claim 2, characterized in that, The step of using a sliding window debouncing mechanism to determine whether a key-value pair is a truly changed key-value pair includes: Within a sliding window of length N simulation steps, count the number of simulation steps in which the state value of the key-value pair continuously appears; If the number of simulation steps reaches a preset step size threshold, the key-value pair is determined to be a key-value pair that has actually changed; otherwise, the key-value pair is determined to have not changed.

4. The method as described in claim 2, characterized in that, The event-based clustering of the state change data forms multiple structured change events, including: The state change data sequence is determined based on each state change data and its corresponding simulation step size number. Starting with the first state change data in the state change data sequence, each state change data in the state change data sequence is traversed sequentially. For the current state change data, calculate the difference between its simulation step size number and the simulation step size number of the previous state change data. If the difference in the sequence number is less than or equal to a preset difference threshold, then the currently traversed state change data will be classified into the current change event. If the difference in the sequence number is greater than the preset difference threshold, the current change event is terminated, and a new change event is created using the current state change data as the starting change data. Structured change events are formed based on each of the aforementioned change events.

5. The method as described in claim 1, characterized in that, The process involves semantically transforming each of the structured change events to generate its corresponding natural language description, and then performing inductive reasoning on each of the natural language descriptions to obtain a sequence of change event descriptions, including: For each structured change event, the state change data contained in the structured change event are converted into their respective natural language descriptions according to a preset logical relationship. The preset logical relationship includes the main object, state name, reference object, old value, and new value. Based on the natural language descriptions, determine the event text, and determine the sequence of change event descriptions based on the event texts.

6. The method as described in claim 1, characterized in that, The pose data includes coordinates and rotation angles. The step of determining the initial object of interest based on the pose data of two adjacent simulation steps includes: For each object, obtain the pose data corresponding to the previous simulation step number and the next simulation step number in the simulation data. Perform a difference operation between the pose data corresponding to the next simulation step number and the pose data corresponding to the previous simulation step number to obtain the position offset and angle offset. If either the position offset or the angle offset exceeds the corresponding preset offset threshold, the object is identified as a preliminary object of interest.

7. The method as described in claim 1, characterized in that, The step of determining neighboring objects based on the distance between the initial object of interest and other objects includes: Calculate the minimum spatial distance between the bounding box of the initial object of interest and the bounding boxes of all other objects; Other objects whose minimum spatial distance is less than or equal to a preset distance threshold are considered as neighboring objects of the initial object of interest.

8. The method as described in claim 1, characterized in that, The step of obtaining the state of each object in the target attention set includes: For each object in the target attention set, obtain the object's own attributes and use the own attributes as the object's self-check status. The object's self-check status includes restricted area entry status, posture data, speed data, opening and closing status, and temperature. For any pair of two objects in the target focus set, obtain the relative relationship between the two objects. The mutual check status between the two objects is determined based on the relative relationship; the mutual check status includes contact status, overlapping status, proximity status and support status.

9. The method as described in claim 1, characterized in that, The linear motion parameters include linear velocity, linear acceleration, and linear jerk, and these parameters are determined in the following manner: For each object in the target focus set, obtain the position sequence of that object within multiple consecutive simulation steps; The linear velocity is determined based on the first derivative of the position sequence with respect to time. The linear acceleration is determined based on the first derivative of the linear velocity with respect to time. The linear jerk is determined based on the first derivative of the linear acceleration with respect to time.

10. An embodied intelligent security assessment device based on state change and event clustering, characterized in that, include: The data acquisition module acquires simulation data from the robot's evaluation task, and based on the simulation data, acquires the pose data of each object in each simulation step; each object includes a first object and a second object, the first object and the second object being any two objects in the simulation environment, and the simulation data is obtained through the simulation environment; The initial object of interest determination module is used to determine the initial object of interest based on the pose data of two adjacent simulation steps. The target attention set determination module is used to determine each neighboring object based on the distance between the initial attention object and other objects, with the initial attention object as the center, and form a target attention set with the initial attention object and each of the neighboring objects. The motion parameter acquisition module is used to acquire the state and linear motion parameters of each object in the target interest set. The linear motion parameters are determined by the pose data of the object in the simulation data of multiple consecutive simulation steps. The state includes self-check state and mutual check state. The state data recording module is used to determine structured state data records based on each of the pose data, each of the linear motion parameters, and each of the states; The event change description sequence determination module is used to determine state change data based on the structured state records, perform event-based clustering on the state change data to form multiple structured change events, perform semantic transformation on each structured change event to generate its corresponding natural language description, and perform inductive reasoning on each natural language description to obtain a change event description sequence; the state change data is used to record a real state change event; the structured change event refers to a structured data container formed by clustering multiple state change data with consecutive simulation step size numbers according to time continuity. The evaluation module is used to evaluate the evaluation task based on the sequence of change events and preset evaluation rules, and obtain an evaluation score.

Images