Data acquisition method, robot and device of embodied intelligent terminal

Abstract

The application provides a data acquisition method, a robot and a device of a body intelligent terminal, wherein the application changes the traditional thinking of data capture dominated by a "perception end / brain end", and defines a high real-time control closed loop end (a cerebellum end) as an arbitration center of global abnormalities. By using the cerebellum end as a unique position of a data intersection point of "physical execution" and "high-level algorithm of the brain end", a three-in-one abnormality monitoring grid of "physical hard abnormality", "spatiotemporal soft divergence" and "human-machine strong occupation" is constructed. In combination with respective ring buffers (Ring Buffer) of the brain end and the cerebellum end, high-precision reverse trigger capture with historical backtracking of long-tail scene data is realized.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and more specifically, to a data acquisition method, robot, and device for an embodied intelligent terminal. Background Technology

[0002] With the rapid development of robotics and autonomous driving technologies, there is an increasing demand for intelligent control terminals that utilize artificial intelligence, multi-sensor fusion, high-precision maps, and real-time decision-making algorithms to perform intelligent control tasks such as environmental perception, path planning, and motion control. To improve the control accuracy of these intelligent control tasks, it is necessary to collect a large amount of real-world scenario data to train the operational algorithms.

[0003] In actual operation, embodied intelligent terminals mostly operate in conventional scenarios, such as those with clear road markings and standardized movement of objects on the road. Data in these scenarios is easily collected on a large scale, and the business algorithms can quickly learn to control the system in such scenarios. However, due to various uncertainties during operation, embodied intelligent terminals inevitably encounter various sporadic and extremely rare abnormal scenarios, such as sudden intrusion of obstacles or hidden obstacles that the business algorithms fail to recognize. These scenarios are statistically located at the "long tail" of the data distribution, hence the term "long tail scenarios." Although long tail scenarios have a low probability of occurrence and are highly random, they can easily lead to operational security issues for embodied intelligent terminals once they occur. Therefore, to improve the business algorithms' ability to cope with long tail scenarios, it is necessary to collect a large amount of real long tail data for iterative training of the business algorithms.

[0004] However, traditional methods for acquiring long-tail data typically fall into two categories: one is to record all data during the operation of the embodied smart terminal, and the other is to collect data when a security incident occurs on the embodied smart terminal in a long-tail scenario. However, both of these methods either result in massive data volume, high bandwidth consumption, and high storage costs, or incomplete and inadequate long-tail data collection, failing to effectively support the iteration of business algorithms. Summary of the Invention

[0005] In view of this, this application provides a data acquisition method, robot, and device for an embodied intelligent terminal to solve the problem of data redundancy caused by the inability to remove non-long-tail data when collecting long-tail data, as well as the problem of incomplete and inadequate data acquisition.

[0006] Specifically, this application is implemented through the following technical solution:

[0007] In a first aspect, embodiments of this application provide a data acquisition method for an embodied intelligent terminal, the embodied intelligent terminal including a processing module for environmental perception and task planning and a control module for real-time motion control; the processing module is provided with a first circular buffer for caching environmental perception data of each frame; the control module is provided with a second circular buffer for caching control data of each frame; the method includes: Using the control module, anomaly detection is performed on the embodied smart terminal based on target data under multiple detection dimensions; the multiple detection dimensions include physical bus dimension, processing module output dimension and interaction dimension, and different detection dimensions correspond to different anomaly scenarios; If any target detection dimension is detected as abnormal, a data acquisition instruction is generated based on the detection time and sent to the processing module. Using the control module, first cached data located within the first time window corresponding to the detection time is collected from the second circular buffer; the first time window is obtained by backtracking a first duration and appending a second duration, respectively, starting from the data frame time corresponding to the detection time; The processing module responds to the data acquisition command, determines a second time window based on the detection time, and acquires second cached data located within the second time window from the first annular buffer. The first cached data, the second cached data, and the scene labels corresponding to the target detection dimension are transmitted to cloud storage.

[0008] Secondly, embodiments of this application also provide a robot, the robot including a processing module for environmental perception and task planning, a control module for real-time motion control, and a transmission module for data transmission; the processing module is provided with a first circular buffer for caching each frame of environmental perception data; the control module is provided with a second circular buffer for caching each frame of control data; wherein: The control module is used to perform anomaly detection on the robot based on target data under multiple detection dimensions. The multiple detection dimensions include physical bus dimension, processing module output dimension, and interaction dimension, and different detection dimensions correspond to different anomaly scenarios. If an anomaly is detected in any target detection dimension, a data acquisition instruction is generated according to the detection time and sent to the processing module. First cached data located within the first time window corresponding to the detection time is collected from the second circular buffer. The first time window is obtained by backtracking a first duration and appending a second duration, starting from the data frame time corresponding to the detection time. The processing module is configured to respond to the data acquisition signal, determine a second time window based on the detection time, and acquire second cached data located within the second time window from the first annular buffer. The transmission module is used to transmit the first cached data, the second cached data, and the scene labels corresponding to the target detection dimension to cloud storage.

[0009] Thirdly, embodiments of this application also provide a data acquisition device for a cloaked intelligent terminal, the cloaked intelligent terminal including a processing module for environmental perception and task planning and a control module for real-time motion control; the processing module is provided with a first circular buffer for caching environmental perception data of each frame; the control module is provided with a second circular buffer for caching control data of each frame; the device includes: The detection unit is used to perform anomaly detection on the embodied smart terminal based on target data under multiple detection dimensions using the control module; the multiple detection dimensions include physical bus dimension, processing module output dimension and interaction dimension, and different detection dimensions correspond to different anomaly scenarios; The generation unit is used to generate a data acquisition instruction based on the detection time if any abnormality is detected in any target detection dimension, and send it to the processing module. The first acquisition unit is used to acquire first cached data located within the first time window corresponding to the detection time from the second circular buffer using the control module; the first time window is obtained by backtracking a first duration and appending a second duration, respectively, starting from the data frame time corresponding to the detection time. The second acquisition unit is used to utilize the processing module in response to the data acquisition command, determine a second time window based on the detection time, and acquire second cached data located within the second time window from the first annular buffer. The transmission unit is used to transmit the first cached data, the second cached data, and the scene labels corresponding to the target detection dimension to cloud storage.

[0010] The data acquisition method, robot, and device for embodied intelligent terminals provided in this application embodiment, by setting up circular buffers in the processing module and the control module respectively, can realize historical caching of various data collected by the two modules, thereby providing a historical data foundation for subsequent data acquisition when anomalies are detected, avoiding the incomplete and incomplete data problems caused by existing technologies that only collect data when a security incident occurs. Utilizing the control module to perform anomaly detection on the embodied intelligent terminal based on target data under multiple detection dimensions, it can not only achieve comprehensive and reliable detection of long-tail scenarios caused by various anomalies, avoiding the problem of missing long-tail scenarios, but also achieve early detection of various anomalies based on the real-time capabilities of the control module, avoiding the incomplete data problems caused by data collection only when a security incident occurs. When an anomaly is detected in any target detection dimension, the control module generates a data acquisition command and sends it to the processing module. Simultaneously, the control module collects data from the second circular buffer according to a first time window. This allows for timely acquisition of control data (i.e., first cached data) before and after the anomaly from the second circular buffer, and timely acquisition of environmental perception data (i.e., second cached data) from the first circular buffer according to a second time window. This ensures the completeness and accuracy of anomaly data acquisition by simultaneously acquiring anomaly-related cached data in both the control and processing modules. By transmitting the collected first and second cached data, along with the scene labels corresponding to the target detection dimensions, to cloud storage, complete uploading of data before and after the anomaly occurs is achieved. Furthermore, the upload process only transmits long-tail scene data related to the anomaly, avoiding redundant data from non-long-tail scenes, effectively reducing the amount of uploaded data, bandwidth usage, and storage costs. Moreover, uploading the scene labels corresponding to the target detection dimensions along with the collected data facilitates cloud-based differentiation of long-tail data from different scenarios based on the scene labels, providing convenience for targeted training of intelligent terminals for different long-tail scenarios.

[0011] Furthermore, the data acquisition method, robot, and device for the embodied intelligent terminal provided in this application embodiment can accurately identify long-tail scenarios with physical anomaly types such as collisions, slippage, and overload by using the changes in motor current and absolute acceleration under the physical bus dimension over multiple consecutive cycles as the basis for anomaly judgment.

[0012] Furthermore, the data acquisition method, robot, and device for the embodied intelligent terminal provided in this application embodiment dynamically determine a first anomaly threshold based on the current adhesion coefficient, working mode, and motor attributes of the embodied intelligent terminal on the current driving road surface. This can accurately determine a first anomaly threshold that matches the actual operating conditions and environment. Then, using this first anomaly threshold for anomaly detection can improve the anomaly detection at the physical bus dimension, enhance robustness to the operating environment and working conditions, and avoid false alarms and missed alarms caused by changes in the environment and working conditions.

[0013] Furthermore, the data acquisition method, robot, and device for the embodied intelligent terminal provided in this application embodiment can also identify long-tail scenes corresponding to abnormal situations such as slow SLAM drift and trajectory tracking disconnection by determining the real-time trajectory error and positioning error, and using the changes in positioning error over multiple consecutive cycles and the magnitude of trajectory error as the basis for anomaly judgment. This improves the accuracy and reliability of anomaly detection in the output dimension of the processing module.

[0014] Furthermore, the data acquisition method, robot, and device for the embodied smart terminal provided in this application can dynamically adjust the buffer depth of the first and second ring buffers based on the real-time moving speed of the embodied smart terminal and the environmental complexity of the environment. This can ensure the complete acquisition of key and valuable long-tail data while controlling the memory occupation of the buffer and improving the utilization efficiency of storage resources.

[0015] Furthermore, the data acquisition method, robot, and device for the embodied smart terminal provided in this application divide the cached data to be uploaded into cached data to be compressed and uncompressed cached data according to the detection time and scene tags. This enables differentiated compression processing of each cached data, ensuring the complete preservation of key data while removing redundancy from non-key data, thereby further reducing data transmission bandwidth and storage costs.

[0016] Furthermore, the data collection method, robot, and device for the embodied smart terminal provided in this application embodiment determine the upload priority of each cached data and upload the cached data according to the upload priority matching method. This can achieve both priority uploading of important cached data with high priority, ensuring that long-tail data of high-value abnormal scenarios can be uploaded to the cloud in a timely manner, and uploading of cached data with low priority when the network allows, reducing the waste of real-time and fast network resources.

[0017] Furthermore, the data acquisition method and robot provided in this application embodiment can also directly realize anomaly detection at the physical bus dimension by using logic chips that are related to the physical bus dimension, in the case of setting up logic chips, thereby improving the anomaly detection speed and reliability to the hardware level.

[0018] Furthermore, the data acquisition method, robot, and device for the embodied smart terminal provided in this application embodiment can also avoid cross-chip communication by dividing a single chip into two different functional domains: a processing module and a control module. This improves communication efficiency, reduces the risk of communication failure, and lowers hardware costs.

[0019] Furthermore, the data collection method, robot, and device for embodied smart terminals provided in this application embodiment can achieve synchronous collection of long-tail data from each embodied smart terminal in the entire terminal cluster by adopting a cloud-based data collection notification method, thereby improving the efficiency and comprehensiveness of long-tail data collection.

[0020] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating a data acquisition method for a personal smart terminal according to an exemplary embodiment of this application; Figure 2 This is a terminal architecture interaction diagram illustrating a data acquisition method according to an exemplary embodiment of this application; Figure 3 This is another flowchart illustrating a data acquisition method in an exemplary embodiment of this application; Figure 4 This is a schematic diagram of the architecture of a robot shown in an exemplary embodiment of this application; Figure 5 This is a schematic diagram of a data acquisition device for a personal smart terminal, as illustrated in an exemplary embodiment of this application; Figure 6 This is a schematic diagram of the structure of a computer device shown in an exemplary embodiment of this application. Detailed Implementation

[0022] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0023] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0024] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0025] Research has revealed that in the practical applications of various embodied intelligent terminals such as robots and autonomous driving terminals, there exist long-tail scenarios with extremely low probability of occurrence, few individual scenario samples, but numerous types and significant cumulative impact. Examples include pedestrians / pets suddenly crossing the road, debris falling from other vehicles, and improperly placed construction barriers. In the data closed-loop system of embodied intelligent terminals and autonomous driving, collecting real-world "long-tail scenario" data for algorithm iteration (such as training large perception models and optimizing SLAM robustness) is a crucial node. The accuracy and comprehensiveness of the collected "long-tail scenario" data directly affect the effectiveness of algorithm iteration. Existing data collection mainly falls into several categories: 1. Full Data Acquisition: During the operation of the embodied intelligent terminal, all raw data streams collected by all sensors (such as cameras and LiDAR) are uploaded to the cloud. While this method ensures that data from all abnormal scenarios (i.e., long-tail scenarios) are not missed, it generates a large amount of redundant data from regular scenarios, resulting in massive local storage consumption, extremely high cloud bandwidth consumption and storage costs. Furthermore, it is quite difficult for algorithm engineers to retrieve truly valuable abnormal long-tail data samples from the full-face data.

[0026] 2. Data Acquisition Triggered by Perception Algorithms: Data acquisition is triggered based on the confidence level of the environmental perception algorithm in the perception module, such as when the perception confidence of the deep learning model decreases. While this method reduces the collection of redundant data, it often results in false positives and false negatives, making it impossible to accurately identify various abnormal events and leading to the loss of a large amount of valuable long-tail data. For example, if the perception algorithm itself misses an obstacle, it will not recognize its presence and therefore will not trigger data acquisition, resulting in the complete loss of high-value collision / near-ground long-tail data.

[0027] 3. Data Acquisition Based on Safety Incidents: Data acquisition is triggered when the chassis of the embodied smart terminal experiences a physical collision or the terminal falls over. This method can only capture long-tail data under limited safety incidents and cannot collect long-tail data in scenarios such as perception algorithm anomalies or hardware malfunctions of the embodied smart terminal itself. Furthermore, this method can only collect residual images and data after a safety incident, and cannot obtain process data before the anomaly occurs, resulting in incomplete long-tail data acquisition and failing to support closed-loop algorithm iteration.

[0028] 4. Timed / distance recording and collection: Data collection is triggered at fixed intervals or after the smart terminal travels a fixed distance. This method not only collects redundant data in non-long-tail scenarios, but also causes the omission of long-tail data in long-tail scenarios, which has obvious drawbacks.

[0029] Based on the above research, this application provides a data acquisition method, robot, and device for embodied intelligent terminals. By setting up circular buffers in the processing module and control module respectively, historical caching of various data collected by the two modules can be achieved, thus providing a historical data foundation for subsequent data acquisition when anomalies are detected. This avoids the incomplete and inadequate data problems caused by existing technologies that only collect data when a security incident occurs. Utilizing the control module to perform anomaly detection on the embodied intelligent terminal based on target data under multiple detection dimensions, it not only achieves comprehensive and reliable detection of long-tail scenarios caused by various anomalies, avoiding the problem of missing long-tail scenarios, but also enables early detection of various anomalies based on the real-time capabilities of the control module, avoiding the incomplete data problems caused by data collection only when a security incident occurs. When an anomaly is detected in any target detection dimension, the control module generates a data acquisition command and sends it to the processing module. Simultaneously, the control module collects data from the second circular buffer according to a first time window. This allows for timely acquisition of control data (i.e., first cached data) before and after the anomaly from the second circular buffer, and timely acquisition of environmental perception data (i.e., second cached data) from the first circular buffer according to a second time window. This ensures the completeness and accuracy of anomaly data acquisition by simultaneously acquiring anomaly-related cached data in both the control and processing modules. By transmitting the collected first and second cached data, along with the scene labels corresponding to the target detection dimensions, to cloud storage, complete uploading of data before and after the anomaly occurs is achieved. Furthermore, the upload process only transmits long-tail scene data related to the anomaly, avoiding redundant data from non-long-tail scenes, effectively reducing the amount of uploaded data, bandwidth usage, and storage costs. Moreover, uploading the scene labels corresponding to the target detection dimensions along with the collected data facilitates cloud-based differentiation of long-tail data from different scenarios based on the scene labels, providing convenience for targeted training of intelligent terminals for different long-tail scenarios.

[0030] The shortcomings of the above solutions are the result of the inventor's practical experience and careful research. Therefore, the discovery process of the above problems and the solutions proposed in this application below should be considered as the inventor's contributions to this application.

[0031] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0032] It is understood that before using the technical solutions disclosed in the various embodiments of this application, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this application in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0033] To facilitate understanding of this embodiment, a data acquisition method for a avatar intelligent terminal disclosed in this application embodiment will first be described in detail. The execution subject of the data acquisition method for a avatar intelligent terminal provided in this application embodiment is generally an avatar intelligent terminal with certain computing capabilities, such as a legged robot, a wheeled robot, an autonomous driving terminal, etc. In some possible implementations, the data acquisition method for the avatar intelligent terminal can be implemented by the processor calling computer-readable instructions stored in the memory.

[0034] The following describes the data collection method of the embodied smart terminal provided in the embodiments of this application, taking the embodied smart terminal as the execution subject as an example.

[0035] like Figure 1 The flowchart shown is a data acquisition method for a personal smart terminal provided in an embodiment of this application, which may include the following steps: S101: Using the control module, anomaly detection is performed on the embodied intelligent terminal based on target data under multiple detection dimensions; the multiple detection dimensions include physical bus dimension, processing module output dimension and interaction dimension, and different detection dimensions correspond to different anomaly scenarios.

[0036] First, the embodied intelligent terminal in this application can be an intelligent hardware capable of autonomous perception, decision-making, action, and feedback in the physical world, such as a wheeled robot, a bipedal robot, a robot, or an autonomous driving terminal. The embodied intelligent terminal can at least include a processing module for environmental perception and task planning, and a control module for real-time motion control.

[0037] Specifically, the processing module acts as the "brain" of the embodied intelligent terminal. It primarily runs non-real-time operating systems (such as Linux / Robot Operating System 2 (ROS2)) and handles high-performance non-real-time high-level computing tasks, such as deep learning perception of environmental data from multiple cameras / LiDAR, large-scene SLAM mapping, and global path planning. The control module acts as the "cerebellum" of the embodied intelligent terminal. It primarily runs high-real-time operating systems (such as Real-Time Operating Systems (RTOS) or Linux systems with RT-Preempt patches) and handles physical bus communication (such as Controller Area Network (CAN) bus communication and Ethernet for Control Automation Technology (EtherCAT) bus communication), chassis motion control algorithms (such as gait control and inverse kinematics solving), and overall system state machine management, thereby achieving real-time control of the embodied intelligent terminal. The processing and control modules can interact via Ethernet or PCIe based on the DDS protocol. For example, the processing module can send SLAM positioning results to the control module, which can then control the embodied smart terminal in real time based on the SLAM positioning results. The processing and control modules can establish high-frequency asynchronous communication based on a time synchronization protocol (such as Precision Time Protocol (PTP) / Generalized Precision Time Protocol (gPTP)) to ensure time synchronization between the two processing modules.

[0038] The processing module may include a first circular buffer for caching environmental perception data for each frame; the control module may include a second circular buffer for caching control data for each frame. The environmental perception data refers to the environmental data collected by sensors deployed on the intelligent terminal, such as point cloud data collected by LiDAR and / or image data collected by a camera. The first circular buffer is a fixed-size, first-in-first-out (FIFO) circular buffer located in the memory corresponding to the processing module. It is frequently rolled over in memory, statically storing environmental perception data for each frame within a certain time range. Optionally, the first circular buffer may also cache physical bus data, control decision data from the processing module, and feedback data from the control module. The control data may include body perception data and control status data acquired by the control module. Body perception data may include inertial measurement unit (IMU) data, torque sensor data, etc.; control status data may include joint status, system status, and diagnostics (such as the operating status, fault codes, and health status of the control module and each execution node) transmitted from the control module to the processing module. The second circular buffer is a fixed-size, FIFO-based circular buffer set up in the memory corresponding to the control module. It is frequently rolled over in memory and statically stores control data for each frame within a certain time range. Optionally, the second circular buffer can also cache physical bus data, such as control commands and movement trajectories sent from the processing module to the cerebellum.

[0039] For example, the processing module can be a high-performance computing unit within the embodied intelligent terminal, used to process environmental perception data such as images and point clouds acquired by the embodied intelligent terminal, and to perform tasks such as scene understanding, Simultaneous Localization and Mapping (SLAM), path planning, and task decision-making. The control module can be a real-time control unit within the embodied intelligent terminal, used to receive task instructions output by the processing module, to perform real-time control tasks such as motion and attitude control and safety protection for the embodied intelligent terminal, and to monitor the operating status of the embodied intelligent terminal in real time during execution.

[0040] In one embodiment, the processing module and the control module can be different chips. For example, the processing module can employ a high-performance chip combining a multi-core Central Processing Unit (CPU) and a high-performance Neural Processing Unit (NPU). This chip can connect to sensors via interfaces such as the Mobile Industry Processor Interface-Camera Serial Interface (MIPI-CSI), Gigabit Multimedia Serial Link (GMSL), or Ethernet. These sensors may include, for example, cameras or LiDAR. The control module can be a System-on-Chip (SoC) with a real-time kernel and connects to motor drivers, high-frequency wheel speedometers / foot odometers, joint encoders, and inertial measurement units (IMUs) via a Controller Area Network (CAN) bus. Data exchange between the processing module and the control module can be achieved via Gigabit Ethernet / Peripheral Component Interconnect Express (PCIe) or other high-speed communication links.

[0041] Alternatively, the processing module and the control module can be different functional domains partitioned from the same chip. In practice, a single main control chip with extremely high computing power can be used, with a hardware hypervisor deployed at the chip's underlying layer. The hypervisor divides the main control chip's hardware resources into two virtualized, hard-isolated functional domains: one as a performance domain to implement the processing module's functions, and the other as a security domain to implement the control module's functions. When the processing and control modules belong to different functional domains of the same chip, they can interact via shared memory. This functional domain partitioning simplifies the dual-chip solution to a single-chip solution, not only reducing costs and increasing efficiency but also eliminating physical cables and interfaces for cross-chip communication, thus preventing communication delays and packet loss caused by loose external cables and electromagnetic interference.

[0042] An abnormal scenario refers to a situation during the operation of a smart terminal that may cause it to deviate from its normal operating state. Because of its low probability of occurrence, it is statistically located at the "long tail" end of the data distribution and can also be called a long-tail scenario. In long-tail scenarios, the environmental perception data and control decision data collected by the processing module, as well as the control data collected by the control module (such as body perception data (e.g., sensor data and safety diagnostic data) and control status data), can all be referred to as long-tail data.

[0043] The detection dimension indicates the data dimension to be detected, with different detection dimensions corresponding to different abnormal scenarios. The target data is the terminal data under each detection dimension, with one set of target data for each detection dimension. Detection dimensions can include at least the physical bus dimension, the processing module output dimension, and the interaction dimension. Specifically, the physical bus dimension is used to detect physical anomalies at the execution layer of the embodied intelligent terminal; the target data under the physical bus dimension can include motor current, IMU data, etc. The processing module output dimension is used to perform spatiotemporal consistency soft divergence detection on the output results of the processing module to determine if there are any software algorithm anomalies in the processing module; the target data under the processing module output dimension can include SLAM positioning location, SLAM positioning error covariance matrix, positioning increment, etc. The interaction dimension is used to detect the interaction between humans or external devices and the embodied intelligent terminal to determine if there are any abnormal situations where humans or external devices forcibly control the embodied intelligent device. External devices can be, for example, the cloud, a host computer, etc.; the target data under the interaction dimension can include various human-computer interaction commands, commands from various external devices, etc. That is, in this application, different types of long-tail scenarios can be determined using target data under different detection dimensions. For example, target data in the physical bus dimension mainly corresponds to physical anomaly scenarios such as collisions, slippage, pitfalls, and overload; the processing module output dimension mainly corresponds to algorithm anomaly scenarios such as SLAM drift and trajectory tracking anomalies; and the interaction dimension mainly corresponds to control takeover anomaly scenarios such as manual takeover and cloud-based emergency braking. When any detection dimension meets the corresponding anomaly judgment, it is considered that the embodied intelligent terminal is in the corresponding long-tail scenario, and long-tail data in that long-tail scenario should be collected.

[0044] An anomaly is indicated in the target data across at least one of multiple detection dimensions, suggesting an anomaly in the embodied intelligent terminal. Anomaly detection of the embodied intelligent terminal is essentially determining whether an anomaly exists in the target data across any detection dimension.

[0045] In practice, for each detection dimension, the control module can monitor in real time whether target data exists within that dimension. If not, monitoring can continue. If it exists, the presence of an anomaly in that dimension can be determined based on the target data and the corresponding anomaly detection method. If yes, the embodied intelligent terminal is confirmed to be abnormal; otherwise, the embodied intelligent terminal is temporarily considered to be normal in that dimension. If it is determined that there is no anomaly in that detection dimension, anomaly monitoring for that dimension can continue.

[0046] Optionally, the control module can be used to periodically acquire target data for each detection dimension over a long period of time, and then a preset judgment algorithm can be used to comprehensively analyze the target data for multiple detection dimensions to determine whether there is any abnormality in the embodied intelligent terminal.

[0047] S102: If any target detection dimension is detected as abnormal, a data acquisition instruction is generated based on the detection time and sent to the processing module.

[0048] Here, the detection time can be the time when an anomaly is detected in a detection dimension; this time can also be called the anomaly trigger time. The target detection dimension is the detection dimension where an anomaly is detected. Different detection dimensions can correspond to different scene labels, and one detection dimension can correspond to at least one scene label. Scene labels are used to characterize the scene type corresponding to the anomaly. For example, scene labels corresponding to the physical bus dimension may include execution layer physical overload, execution layer physical collision, slippage, etc.; scene labels corresponding to the processing module output dimension may include spatiotemporal disjointness, SLAM positioning divergence, high tracking error, etc.; and scene labels corresponding to the interaction dimension may include manual forced takeover, emergency braking, suspected perception miss, etc.

[0049] The data acquisition command instructs the processing module to retrieve cached data from the first circular buffer. This command may include at least the detection time. Optionally, the data acquisition command may also include the scene label corresponding to the target detection dimension.

[0050] In the software layer of the control module, an independent and resident trigger manager is set up, such as the shadow data trigger manager. This trigger manager is used to poll the detection results under each detection dimension at a high frequency.

[0051] In practice, once an anomaly is detected in the target data of any detection dimension indicating an anomaly in the embodied intelligent terminal, an anomaly signal can be generated. The trigger manager can promptly monitor the anomaly signal through high-frequency polling, then use the detection dimension corresponding to the anomaly signal as the target detection dimension, and the generation time of the anomaly signal as the detection time. Then, based on the detection time, the trigger manager can generate a data acquisition command and send it to the processing module. Simultaneously, the trigger manager can generate a local snapshot command, which carries the detection time, to trigger the control module to retrieve cached data from the local second circular buffer.

[0052] S103: Using the control module, first buffer data located within the first time window corresponding to the detection time is collected from the second circular buffer area; the first time window is obtained by backtracking the first duration and appending the second duration, respectively, starting from the data frame time corresponding to the detection time.

[0053] The second circular buffer is a First-In-First-Out (FIFO) buffer deployed in the local memory of the control module. It stores the body perception data (such as sensor data and safety diagnostic data) and control status data acquired by the control module in the recent period. The control status data may include, but is not limited to: motor current, IMU three-axis absolute acceleration, IMU angular velocity, wheel speed / joint angular velocity, tracking error, and cost function value output by Model Predictive Control (MPC). The data in the second circular buffer is cached in discrete data frames, and each data frame has a data frame time.

[0054] The first time window can be a data extraction window determined based on the detection time. This window includes portions of time before and after the detection time, ensuring that complete data before and after the occurrence of the anomaly at the detection time can be collected. The first cached data refers to the data frames (i.e., control data of each frame) located within the first time window in the second circular buffer. This data is the control data when the anomaly occurs corresponding to the target detection dimension. Both the first and second durations can be determined based on the operating conditions of the embodied intelligent terminal. For example, the first duration can be 5 seconds, and the second duration can be 10 seconds.

[0055] In practical implementation, the control module can determine the target data frame closest to the detection time based on the detection time and the data frame times of each data frame currently stored in the local second circular buffer. Starting from the data frame time of this target data frame, a second time window is obtained by advancing a first time period forward and appending a second time period backward. The cached data within the appended second time period can be cached after an anomaly in the target detection dimension is detected, while the cached data within the advance first time period can be frozen after an anomaly in the target detection dimension is detected. Then, the control module can collect first cached data related to the anomaly in the target detection dimension from the second circular buffer according to the first time window.

[0056] S104: The processing module responds to the data acquisition command, determines the second time window based on the detection time, and acquires the second buffer data located within the second time window from the first circular buffer.

[0057] Here, under the condition of time synchronization between the processing module and the control module, the time range corresponding to the second time window is consistent with the time range corresponding to the first time window. When there are occasional minor errors in the time synchronization between the processing module and the control module, the time range corresponding to the second time window may have a minor error compared to the time range corresponding to the first time window. The time synchronization method between the processing module and the control module can be: underlying hardware-level clock synchronization + application-layer unified global timestamp mapping. Specifically, the PTP / gPTP protocol can be used to achieve distributed clock synchronization between the processing module and the control module through Gigabit Ethernet. Since the control module has higher real-time performance and minimal clock jitter, it can usually be selected as the master clock source and the processing module as the slave clock source. The time between the processing module and the control module is continuously calibrated through the hardware timestamp of the network interface card's physical layer (PHY) to keep the system and error at both ends at the microsecond level. When data is written to the circular buffer, when the sensing module in the processing module (such as the MIPI-CSI / Gigabit Multimedia Serial Link (GMSL) driver layer) and the body motion module in the control module (such as the IMU / CAN driver layer) receive the raw data, they must read the global timestamp synchronized by gPTP at the first moment in the interrupt service routine (ISR) or the underlying driver, and write the global timestamp as the unique metadata into the structure of each frame of data in the circular buffer. Local relative time is not allowed to be used, ensuring that the time source of each frame written to the circular buffer comes from the main time source.

[0058] The second cached data is the data frames (i.e., environmental awareness data of each frame) located in the second time window within the first circular cache area. This data is the environmental data when the anomaly occurs in the target detection dimension.

[0059] In practice, the processing module can receive and respond to the data acquisition command, determine the second time window in the first ring buffer according to the detection time, and collect the second buffer data located in the second time window from the first ring buffer.

[0060] S105: Transmit the first cached data, the second cached data, and the scene labels corresponding to the object detection dimensions to cloud storage.

[0061] In practice, for anomalies corresponding to the target detection dimension, the first cached data collected by the control module, the second cached data collected by the processing module, and the scene labels corresponding to the target detection dimension can be uploaded to the cloud for storage. This allows for subsequent algorithm iteration based on the first and second cached data for long-tail scenarios corresponding to various scene labels stored in the cloud.

[0062] For example, the control module can be used to transmit the first cached data and the scene labels corresponding to the target detection dimension to cloud storage. Furthermore, the processing module can be used to transmit the second cached data and the scene labels corresponding to the target detection dimension to cloud storage. For the same detected anomaly, the control module and processing module can asynchronously upload the corresponding cached data. Since the scene labels transmitted by both modules are consistent, even with asynchronous uploads, the first cached data, the second cached data, and the anomaly can be aligned based on the scene labels.

[0063] Optionally, after the control module collects the first cached data and the processing module collects the second cached data, the trigger manager can be used to automatically inject abnormal tags into the first cached data and the second cached data according to the scene tags corresponding to the target detection dimension. Then, the trigger manager can be used to upload the first cached data, the second cached data, and the abnormal tags to the cloud for storage.

[0064] In this way, by using the control module as the arbitration center for data acquisition, this application can realize the timely triggering and capture of long-tail data corresponding to various abnormal scenarios that occur in the embodied intelligent terminal, and improve the comprehensiveness of data capture by triggering the capture of data before and after the occurrence of the abnormality in two circular buffers.

[0065] In one embodiment, an anomaly monitoring grid can be pre-built in the control module, including monitoring methods corresponding to each detection dimension. The control module can detect various anomalies in a timely manner through the monitoring methods corresponding to each detection dimension. Specifically, for the above S101, it can be implemented according to the following steps: S101-1: For the physical bus dimension, in the absence of logic chips related to the physical bus dimension, the control module is used to obtain the first abnormal threshold under the physical bus dimension, as well as the motor current and the absolute acceleration output by the inertial measurement unit in multiple consecutive cycles.

[0066] Here, the monitoring method corresponding to the physical bus dimension in the anomaly detection network can be the execution-layer physical anomaly monitoring method. This method is used to detect whether the embodied intelligent terminal has anomalies such as collisions, slippage, or overloads. The logic chip related to the physical bus dimension can be a separate chip, which can be used to execute the monitoring method corresponding to the physical bus dimension. The target data under the physical bus dimension are the motor current and the absolute acceleration output by the inertial measurement unit.

[0067] The first abnormal threshold is used to determine whether there are abnormalities in the motor current and absolute acceleration. The cycle frequency can be set empirically, and this embodiment does not limit it; for example, it can be 100 Hz, i.e., 10 ms in duration. The number of consecutive cycles can be set empirically, and this embodiment does not limit it specifically; for example, 3 consecutive cycles.

[0068] In practical implementation, for the physical bus dimension, the control module can be used to determine whether the intelligent terminal is equipped with logic chips related to the physical bus dimension; if not, the execution layer physical anomaly monitoring method in the control module can be used to obtain the first anomaly threshold under the physical bus dimension in each cycle of multiple consecutive cycles, as well as read the motor current fed back by the chassis CAN bus and the three-axis absolute acceleration of the direct-connected IMU.

[0069] S101-2: Determine whether there is an anomaly at the physical bus level based on motor current, absolute acceleration, and the first anomaly threshold.

[0070] In practical implementation, the control module can be used to determine whether an anomaly exists at the physical bus level by analyzing the relationship between the mutation rate of motor current and a first anomaly threshold across multiple consecutive cycles, as well as the relationship between the mutation rate of absolute acceleration and the first anomaly threshold across multiple consecutive cycles. For example, if the mutation rate of motor current and the mutation rate of absolute acceleration in any cycle are both greater than the first anomaly threshold, an anomaly can be determined at the physical bus level. If the mutation rates of motor current and absolute acceleration in each cycle are not greater than the first anomaly threshold, no anomaly can be determined at the physical bus level.

[0071] In one embodiment, the first abnormal threshold may specifically include a current abnormal threshold for the motor current. and acceleration anomaly threshold for absolute acceleration Regarding S101-2 above, the following steps can be taken: S101-2-1: Using the control module, determine the current change rate of the motor current and the acceleration change rate of the absolute acceleration over multiple consecutive cycles.

[0072] In practice, for each of several consecutive cycles, the motor current in that cycle can be calculated using the physical anomaly monitoring method at the execution layer in the control module. I ( t The derivative of ) This derivative is used as the current mutation rate for that cycle. Simultaneously, the absolute acceleration for that cycle can be calculated using the physical anomaly monitoring method at the execution layer of the control module. a ( t The derivative of ) The absolute value of the derivative As the acceleration mutation rate under that period.

[0073] S101-2-2: If the current mutation rate is greater than the current anomaly threshold for multiple consecutive cycles, and / or the acceleration mutation rate is greater than the acceleration anomaly threshold for multiple consecutive cycles, an anomaly is determined to exist in the physical bus dimension.

[0074] In practice, for each cycle, the data under that cycle can be... With current abnormal threshold Compare and determine Is it greater than In multiple consecutive cycles All greater than In this case, an anomaly can be identified at the physical bus level, specifically a current anomaly, and the scenario label can be determined as execution layer physical overload and / or execution layer physical collision. Similarly, for each cycle, the scenario under that cycle can be... With acceleration anomaly threshold Compare and determine Is it greater than In multiple consecutive cycles All greater than In this case, it can be determined that there is an anomaly in the physical bus dimension, which is an acceleration anomaly, and the scene label is determined to be the slippage label.

[0075] Understandably, the execution-level physical anomaly monitoring method can detect anomalies in multiple consecutive cycles. > and multiple consecutive cycles > When at least one of the two judgment conditions is triggered, it can be determined that the chassis has experienced a physical impact, overload, or slippage, confirming an anomaly in the physical bus dimension, and thus indicating an anomaly in the embodied intelligent terminal. The time at which the triggered condition is detected is taken as the detection time corresponding to the anomaly. Then, the anomaly monitoring grid can immediately send an anomaly signal to the trigger manager so that the trigger manager can promptly detect the anomaly in the physical bus dimension. When both judgment conditions are triggered, the detection time when either judgment condition is triggered can be taken as the detection time corresponding to the anomaly. When both judgment conditions are triggered, the scene label can simultaneously include the execution layer physical overload label, the execution layer physical collision label, and the slippage label.

[0076] In one embodiment, since the operating conditions of the smart terminal may differ, the required first anomaly threshold may also differ. Therefore, in order to improve the accuracy of anomaly determination in the execution layer physical anomaly monitoring method, the step of "obtaining the first anomaly threshold" in S101-1 above can be implemented according to the following steps A1~A3: A1: Using the control module, determine the current adhesion coefficient of the road surface on which the embodied intelligent terminal is traveling, based on the motion state of the embodied intelligent terminal.

[0077] Here, motion state is used to characterize the current motion state of the embodied intelligent terminal. This state can be represented by different data depending on the type of the embodied intelligent terminal. For example, in the case of a legged robot, the running state can be the currently active motion control gait of the legged robot; in the case of a wheeled robot / autonomous driving terminal, the running state can be the tire slip ratio.

[0078] The driving surface is the road surface where the embodied intelligent terminal is currently running. The current adhesion system is the adhesion coefficient provided by the driving surface to the embodied intelligent terminal. The current adhesion coefficient can characterize the type of ground that the embodied intelligent terminal is dealing with, such as asphalt ground, snow and ice ground, mud and grass ground, cement road surface, etc.

[0079] In practice, the control module can be used to obtain the current motion state of the intelligent terminal. Then, based on the current operating state, the equivalent comprehensive adhesion coefficient between the chassis and the road surface can be estimated in real time, and this adhesion coefficient can be used as the current adhesion coefficient.

[0080] A2: Obtain the abnormal threshold to be used from the pre-set threshold calibration table, based on the current adhesion coefficient, the working mode of the smart terminal, and the motor attributes; the threshold calibration table carries the correlation between various abnormal thresholds and the adhesion coefficient, working mode, and motor attributes.

[0081] Here, the operating mode indicates the current operating mode of the embodied smart terminal, such as running mode, creep mode, inspection mode, high-speed mode, etc. Motor attributes characterize the motor hardware parameters of the drive motor configured in the embodied smart terminal, which may include, for example, the inductance, resistance, rated / peak current, etc. of the drive motor.

[0082] The threshold calibration table can be a pre-calibrated multidimensional parameter mapping table, which can store the correlation between different adhesion coefficients, different operating modes, and different motor attributes and the first abnormal threshold. The correlation can be a correspondence between various parameters; for example, adhesion coefficient 1, operating mode 1, and motor attribute 1 are associated with the first abnormal threshold 1, and adhesion coefficient 2, operating mode 2, and motor attribute 2 are associated with the first abnormal threshold 2. The correlation can also be a multinomial fitting function corresponding to various parameters; for example, a multinomial fitting function between the adhesion coefficient, operating mode, and motor attribute and the abnormal threshold.

[0083] The threshold to be used can be a first abnormal threshold that matches the embodied smart terminal under the current attachment coefficient.

[0084] In practice, the control module can be used to determine the working mode and motor attributes of the smart terminal. Then, according to the working mode, motor attributes, current adhesion coefficient, and various relationships stored in the threshold calibration table, the matching abnormal threshold to be used can be determined. The abnormal threshold to be used can include the abnormal current threshold and the abnormal acceleration threshold to be used.

[0085] In one embodiment, the threshold calibration table can be pre-calibrated through the following steps: The core physical significance of the abnormal threshold at the physical bus level lies in capturing the extreme moments of the instantaneous disappearance of the back electromotive force of the drive motor or the abnormal surge in load. Since different drive motors have variations in inductance, resistance, and rated / peak current range, they need to be calibrated individually based on their hardware parameters. Furthermore, the different adhesion coefficients of the road surface mean that conditions such as tire jamming and slippage also need to be considered separately to achieve reasonable calibration. In actual deployment, the calibration parameters in the threshold calibration table can usually be calibrated by collecting data through a closed-loop process. The specific calibration process is as follows: Step 1: First, determine the upper limit of the background noise of the embodied intelligent terminal under various working modes, motor attributes, and adhesion coefficients during normal operation. Allow the embodied intelligent terminal to operate under all permissible extreme conditions (e.g., driving at maximum speed on a rough gravel road, sudden stop at full speed under full load, crossing speed bumps of a specified height, and making full-force turns on grass). Then, collect all the data generated by the embodied intelligent terminal under these various conditions. Data and The data is collected, and based on this data, probability distribution curves corresponding to motor current and absolute acceleration are determined. Then, based on the probability distribution curves, the maximum peak value P_normal_max1 of the motor current and the maximum peak value P_normal_max2 of the absolute acceleration under normal operating conditions can be determined. An abnormal threshold for the motor current is set to... =P_normal_max1 And set the anomaly threshold corresponding to absolute acceleration to =P_normal_max2 .in, and These are weighting coefficients corresponding to the motor current and absolute acceleration, respectively. These coefficients are typically greater than 1 to provide a margin against accidental touches. For example, and Both are 1.3, thus providing a 30% margin against accidental touches.

[0086] Step 2: Compile test data under fault injection (i.e., anomaly injection) and real-world drop scenarios. In a controlled laboratory environment, artificially create hard faults (such as jamming the chassis wheels with a rigid obstacle, or causing the embodied intelligent terminal to blindly crash into a rigid wall at the minimum trigger protection speed), and record these destructive moments. and The data spike curve is obtained, and the fault peak P_fault corresponding to the data spike curve must be much larger than the abnormal threshold determined in step 1. If there is an overlap between P_fault and the abnormal threshold determined in step 1, it indicates that the sensor accuracy is insufficient or that filtering needs to be introduced to improve the rationality and accuracy of the determined abnormal threshold, thereby obtaining the abnormal threshold under various operating modes, motor properties, and adhesion coefficients.

[0087] Step 3. Abnormal thresholds under various operating modes, motor attributes, and adhesion coefficients can be stored in a threshold calibration table within the control module to record different functional modes and different ground surfaces (asphalt / snow / mud / grass). and .

[0088] Thus, based on the offline calibration steps 1 to 3 above, the abnormal thresholds calibrated under various adhesion coefficients, working modes and motor attributes can be obtained. When the control module performs abnormal judgment at the physical bus dimension, the matching first abnormal threshold can be dynamically determined by table lookup and online feedforward correction.

[0089] A3: If the current adhesion coefficient is different from the previously determined historical adhesion coefficient, adjust the anomaly threshold to be used based on the current adhesion coefficient and the historical adhesion coefficient to obtain the first anomaly threshold.

[0090] Here, by determining the abnormal threshold using an offline calibrated threshold calibration table, a preset, fixed abnormal threshold can be obtained. However, due to various changes during the operation of the intelligent terminal, this application also proposes a composite mechanism of "offline benchmark calibration + online feedforward correction" to further improve the accuracy of the abnormal threshold. This mechanism dynamically adjusts the threshold based on the current adhesion coefficient and the threshold calibration table. The historical adhesion coefficient can be the adhesion coefficient determined in the previous cycle of the current cycle.

[0091] In practice, after the control module determines the current adhesion coefficient, it can simultaneously obtain the previously determined historical adhesion coefficient. If no previously determined historical adhesion coefficient exists, the anomaly threshold to be used can be directly used as the first anomaly threshold. If a previously determined historical adhesion coefficient exists, it can be determined whether the current adhesion coefficient differs from the previously determined historical adhesion coefficient. If not, the anomaly threshold to be used can be directly used as the first anomaly threshold. If so, online feedforward correction of the anomaly threshold to be used can be determined. The correction method can be to determine the adjustment range of the anomaly threshold to be used based on the rate of change between the current adhesion coefficient and the historical adhesion coefficient, and adjust the anomaly threshold to be used according to this range to obtain the final first anomaly threshold.

[0092] For example, the anomaly monitoring grid of the control module can pre-store a continuous multinomial fitting function between the "adhesion coefficient and safety clearance" derived from dynamic characteristics to determine the anomaly threshold to be used. When the embodied intelligent terminal moves from a high-adhesion area (such as asphalt road surface) to a low-adhesion condition (such as icy / slippery ceramic tile road surface), the control module can adjust the decision thresholds of the current anomaly threshold and the acceleration anomaly threshold to be used in a feedforward manner according to the rate of change of the adhesion coefficient, so as to improve the sensitivity of capturing nonlinear micro-slip faults under low adhesion, and fundamentally eliminate the risk of missed and false alarms of traditional fixed thresholds in long-tailed heterogeneous terrain.

[0093] Optionally, when adjusting the threshold for abnormal use based on the current adhesion coefficient and the previously determined historical adhesion coefficient, at least one of the threshold for abnormal use current and the threshold for abnormal use acceleration can be adjusted.

[0094] In this way, this application changes the perception triggering logic of the processing module. The control module constructs a three-in-one anomaly monitoring grid of physical hard anomalies, spatiotemporal soft divergence, and human-machine strong preemption. By judging anomalies through methods such as sudden changes in motor current, IMU impulse response, Mahalanobis distance residual, and control preemption, timely detection and data collection of various long-tail scenarios can be achieved.

[0095] S101-3: For the output dimension of the processing module, the control module uses the target movement trajectory output by the processing module and the positioning increment within the preset time window to determine whether there are any anomalies in the output dimension of the processing module.

[0096] Here, the monitoring method corresponding to the output dimension of the processing module in the anomaly detection network can be the spatiotemporal consistency soft divergence monitoring method. This method is used to detect whether there are anomalies such as SLAM positioning drift, control divergence, or large tracking errors in the output results of the processing module of the embodied intelligent terminal. The target movement trajectory can be the globally planned trajectory planned by the processing module for the embodied intelligent terminal. The time range corresponding to the preset time window can be set empirically, and this application embodiment does not impose any limitations. For example, the preset time window can be a time range of 200ms. Each cycle can correspond to a preset time window, and the preset time window corresponding to any cycle can be obtained by tracing back the time range corresponding to the preset time window from the time corresponding to that cycle as the end point. Positioning increment This is used to instruct the processing module to calculate and feed back the SLAM localization algorithm's algorithmic positioning increment within a preset time window in each cycle. The target data in the output dimension of the processing module is the target's movement trajectory and the positioning increment within the preset time window.

[0097] In practical implementation, for the output dimension of the processing module, the spatiotemporal consistency soft divergence monitoring method in the control module can be used to acquire the target movement trajectory issued by the processing module and the positioning increment within a preset time window in each of multiple consecutive cycles. Then, based on the trajectory error between the target movement trajectory and the real-time movement trajectory of the embodied intelligent terminal, and the positioning increment and the real-time displacement increment of the embodied intelligent terminal within the preset time window, it can be determined whether there is an anomaly in the output dimension of the processing module. For example, if the trajectory error in at least one of the multiple consecutive cycles is greater than a preset error, and / or the difference between the real-time displacement increment and the positioning increment in multiple consecutive cycles is greater than a preset value, then it can be determined that there is an anomaly in the physical bus dimension.

[0098] In one embodiment, S101-3 described above can be implemented according to the following steps: S101-3-1: Using the control module, obtain the second abnormal threshold in the output dimension of the processing module, and determine the trajectory error of the embodied intelligent terminal based on the real-time motion trajectory of the embodied intelligent terminal and the target movement trajectory.

[0099] Here, the second anomaly threshold is used to determine whether the positioning increment output by the processing module is abnormal. The real-time motion trajectory can be the actual trajectory of the embody intelligent terminal in each cycle under the real-time control of the control module. The trajectory error characterizes the difference between the actual motion trajectory of the embody intelligent terminal and the motion trajectory expected by the processing module; a larger error indicates a more severe control divergence problem in the processing module.

[0100] In practical implementation, regarding the output dimension of the processing module, in each of multiple consecutive cycles, the control module can acquire the second anomaly threshold and the real-time motion trajectory of the embodied intelligent terminal in that cycle. Then, the trajectory error between the real-time motion trajectory in that cycle and the target motion trajectory output by the processing module can be calculated using a spatiotemporal consistency soft divergence monitoring method. .

[0101] Optionally, in each of several consecutive cycles, the control module can be used to monitor in real time the model error originating from the cost function of the Model Predictive Control (MPC) solver. ,in, It can characterize the confidence level of the target movement trajectory output by the processing module.

[0102] S101-3-2: Determine the target displacement increment of the embodied intelligent terminal within the preset time window based on the target's moving speed within the preset time window.

[0103] Here, the target movement speed can be the movement speed calculated by the control module for the embody intelligent terminal. Target displacement increment. Used to characterize the local physical displacement increment of an embodied smart terminal within a preset time window.

[0104] In practical implementation, a control module can be used to obtain the target moving speed of the embodied intelligent terminal within a preset time window. For example, for a wheeled robot, the outputs of the high-frequency wheel speedometer and IMU of the embodied intelligent terminal within the preset time window can be used as the target moving speed. Then, dynamic pre-integration can be performed on the target moving speed to obtain the speed of the embodied intelligent terminal within the preset time window. .

[0105] S101-3-3: Determine the positioning error of the embodied intelligent terminal based on the target displacement increment and positioning increment.

[0106] Here, the positioning error can be characterized using the Mahalanobis distance residual.

[0107] In practice, the positioning error can be determined according to the following formula: (Formula 1) in, This represents the positioning error (i.e., the Mahalanobis distance residual). This represents the error covariance matrix of the algorithm positioning sent by the processing module in each cycle; This represents the error covariance matrix of the control module during dynamic pre-integration in each cycle. express and The transpose of the difference matrix between them. express and The inverse of the sum matrix.

[0108] In each cycle, the processing module can simultaneously send an error covariance matrix of the SLAM positioning algorithm when sending the target movement trajectory and the SLAM positioning increment. The control module operates in each time cycle. At the same time, the error covariance matrix of a physical integral can be calculated simultaneously. .

[0109] S101-3-4: If the trajectory error is greater than the preset error, and / or the positioning error is greater than the second abnormal threshold in multiple consecutive cycles, it is determined that there is an abnormality in the output dimension of the processing module.

[0110] Here, the preset error can be set based on experience, and this application embodiment does not impose specific limitations. When the trajectory error surges beyond the physical and dynamic limit of the preset error in a short period of time, it indicates that the embodied intelligent terminal is no longer able to travel according to the predetermined trajectory, and it has encountered a risk of loss of control.

[0111] In practice, for each cycle, the trajectory error in that cycle can be compared with a preset error. If the trajectory error in any cycle is greater than the preset error, an anomaly is determined in the output dimension of the processing module, i.e., an anomaly exists in the embodied intelligent terminal. This anomaly is a tracking anomaly, and the scene label is determined as the trajectory tracking anomaly label. The time when the tracking anomaly is determined is used as the detection time. Similarly, for each cycle, if the positioning error is determined to be greater than the second anomaly threshold in multiple consecutive cycles, an anomaly is determined in the output dimension of the processing module. This anomaly is SLAM drift, and the scene label is determined as the SLAM drift label. The time when the SLAM drift is determined is used as the detection time.

[0112] Optionally, the control module obtains the data for each cycle. In this case, it is also possible to do so in multiple consecutive cycles. If all values ​​are greater than the preset value, it is determined that there is an anomaly in the output dimension of the processing module. This anomaly is the algorithm anomaly of the processing module, and the scene label is determined as the algorithm cost anomaly label. The time when the algorithm anomaly of the processing module is detected is taken as the detection time.

[0113] Understandably, the spatiotemporal consistency soft divergence monitoring method determines that the trajectory error in any period is greater than the preset error, the positioning error is greater than the second anomaly threshold in multiple consecutive periods, and the error is greater than the second anomaly threshold in multiple consecutive periods. When at least one of the three preset judgment conditions is triggered, it can be determined that the processing module has an algorithm soft divergence problem, which also indicates that the embodied intelligent terminal is abnormal. Then, the anomaly monitoring grid can immediately send an anomaly signal to the trigger manager so that the trigger manager can promptly detect the anomaly in the output dimension of the processing module. When all three judgment conditions are triggered, the detection time when any judgment condition is triggered can be used as the detection time corresponding to the anomaly. When all three judgment conditions are triggered, the scene label can simultaneously include tracking anomaly, SLAM drift, and algorithm anomaly labels.

[0114] In this way, by considering the relationship between the positioning error and the second anomaly threshold over multiple consecutive cycles (e.g., 3 cycles), signal stabilization can be achieved, reducing the impact of transient noise. This also avoids excessively long timelines caused by excessively long reconfirmation periods to check for anomalies in the processing module's output dimension, leading to severe response delays. The cycle frequency is typically 10ms. Since the movement of the intelligent terminal over 3 consecutive cycles (30ms) is usually relatively short, there is ample time to confirm anomalies and perform emergency braking or data acquisition.

[0115] In one embodiment, the step of "obtaining the second abnormal threshold" in S101-3-1 above can be implemented according to the following steps B1~B2: B1: Using the control module, based on the movement pattern of the smart terminal, obtain the candidate threshold pre-calibrated for positioning error.

[0116] Here, the mode of movement is used to indicate the carrier on which the embodied intelligent terminal relies for movement, and it is determined according to the terminal type of the embodied intelligent terminal. For example, the mode of movement for a wheeled robot / autonomous driving terminal can be wheel-driven; the mode of movement for a legged robot can be alternating legged drive.

[0117] Candidate thresholds can be pre-standardized positioning error thresholds, and different movement methods can correspond to different baseline thresholds.

[0118] In practice, the control module can be used to determine the movement mode of the smart terminal, and then, according to the movement mode, candidate thresholds that match the current movement mode can be selected from the pre-defined baseline thresholds corresponding to various movement modes.

[0119] The core physical meaning of Mahalanobis distance lies in measuring the deviation between the "algorithm positioning increment" and the "physical motion integral," specifically, how many standard deviations it has been stretched to. Mathematically, if the errors of both follow a Gaussian distribution, then the square of the Mahalanobis distance will strictly follow a chi-square distribution. Based on this, the baseline thresholds for different movement methods can be calibrated using the following steps: For embodied intelligent terminals, the 2D pose (Delta x, Delta y, Deltatheta) has three degrees of freedom. Delta x and Delta y represent the displacement increments in the horizontal and vertical directions, respectively, while Deltatheta represents the change in angle. If the control module is to avoid false alarms within a 99% confidence interval (i.e., tolerate 99% statistical fluctuation under normal conditions), according to the chi-square distribution table, when the degrees of freedom are 3 and P=0.99, the chi-square critical value is 11.34. Therefore, the theoretical baseline threshold Threshold_1 can be... ≈3.37. However, in actual engineering deployments, the above theoretical benchmark threshold is a theoretical derivation based on the assumption of "ideal Gaussian white noise." In the complex operating conditions of actual embodied intelligent terminals, this assumption contains a large number of unmodeled nonlinear disturbances; therefore, the benchmark threshold must be corrected in conjunction with the movement mode. Considering the pre-integration results of the high-frequency wheel speedometer and IMU when the wheeled robot travels on flat ground... Very smooth, uncertainty covariance matrix Since the threshold is relatively small, in order to improve the accuracy of SLAM positioning drift detection, the theoretical benchmark threshold for wheeled robots can be lowered. For example, the benchmark threshold for wheeled robots can be adjusted to 3.5.

[0120] Legged robots, whether quadruped or bipedal, inherently exhibit non-Gaussian, multi-peak noise in their physical integral due to frequent footstep impacts, high-frequency minute slippage, and complex joint dynamic force measurement noise. To prevent legged robots from erroneously triggering degradation due to footstep vibrations during trotting or obstacle crossing, the theoretical baseline threshold for legged robots must be significantly amplified. For example, the baseline threshold for legged robots can be determined to be any threshold between 4.5 and 5.5.

[0121] B2: Based on the motion state of the embodied smart terminal and the environmental characteristics of the current environment of the embodied smart terminal, the candidate threshold is adjusted to obtain the second abnormal threshold.

[0122] Here, motion state is used to characterize the linear velocity and angular velocity of the embodied intelligent terminal. Environmental characteristics are used to characterize the complexity and feature richness of the environment in which the embodied intelligent terminal is located.

[0123] When the operating status indicates that the embodied intelligent terminal is in a low-speed state, the impact on spatial displacement is negligible due to the spatial-temporal alignment delay of the sensors, resulting in a very small consistency residual. Therefore, the baseline threshold can be directly used as the second anomaly threshold. However, when the operating status indicates that the embodied intelligent terminal is traveling at high speed or spinning rapidly, the SLAM calculation delay of the processing module and the data transmission delay of the control module both lead to… and Even a slight time lag misalignment in the time window alignment can cause a surge in Mahalanobis distance after linear amplification. Therefore, a correction coefficient positively correlated with the motion state can typically be introduced to adjust the baseline threshold. For example, the formula for adjusting the baseline threshold can be as follows: (Formula 2) in, This represents the outlier threshold after adjusting for the baseline threshold (i.e., the candidate threshold). This represents the baseline threshold (i.e., the candidate threshold). and This represents the preset weighting coefficients. Represents linear velocity v The absolute value, Represents angular velocity w The absolute value of . Thus, since the timestamp alignment residual between the processing module and the control module diverges linearly with the intensity of motion under high dynamic conditions, the uncertainty envelope under high-speed conditions can be broadened by introducing a velocity gain factor to dynamically compensate for the time-varying reference threshold.

[0124] Meanwhile, regarding environmental characteristics, for environments with rich features (such as standard offices, factory workshops, etc.), the processing module issues... The values ​​are typically small, therefore the denominator of the Mahalanobis distance is small and the sensitivity is high, allowing the baseline threshold to be directly used as the required anomaly threshold. In environmentally degraded scenarios (such as diffuse reflective glass curtain walls, long corridors without environmental features, large-scale strong winds and sandstorms, direct strong sunlight, etc.), the processing module exhibits deterministic algorithmic "soft divergence." If a highly sensitive baseline threshold is maintained, the embodied intelligent terminal may be unable to traverse environmentally degraded scenarios. Therefore, when the global positioning confidence of the processing module (i.e., ...) is low... When the trace or determinant continues to increase, the control module needs to dynamically increase the baseline threshold, allowing the embodied intelligent terminal to tolerate a wider range of slow drifts, and temporarily handing over the defense focus to the local reactive obstacle avoidance of the control module's underlying shadow planner.

[0125] In practical implementation, the control module can be used to acquire the real-time motion status of the embodied intelligent terminal, such as linear velocity and angular velocity, based on the measurement hardware on the embodied intelligent terminal (such as a high-frequency wheel speed meter / foot-based odometer, IMU, etc.). Simultaneously, the covariance of SLAM positioning can be obtained from the processing module. The environmental characteristics of the current environment of the intelligent terminal are determined. Then, based on the real-time motion state, a nonlinear time-delay correction based on Formula 2 above can be injected. According to the linear velocity and angular velocity indicated by the real-time motion state, Formula 2 above is used to initially adjust the candidate threshold to obtain an intermediate threshold. Then, scene adaptive convergence can be performed according to the environmental characteristics to adjust and obtain the final second anomaly threshold. For example, the global covariance matrix output in real time by the SLAM algorithm of the processing module can be combined. The evolution trend of eigenvalues ​​can be used to determine whether degradation due to environmental characteristics has led to [the following]. In the event of abnormal expansion, if it exists, the control layer decision state machine of the control module can increase the intermediate threshold in a coordinated manner to obtain a second abnormal threshold that finally matches the environmental characteristics. This allows the control module to adaptively identify "slippage in normal algorithm confidence caused by changes in environmental characteristics" and "chassis nonlinear uncontrolled divergence," significantly improving the operational robustness of the embodied intelligent terminal in long-tailed heterogeneous working conditions while ensuring physical safety. If it does not exist... In the case of abnormal expansion, the intermediate threshold can be directly used as the second abnormal threshold.

[0126] Optionally, after obtaining the candidate threshold, it can be adjusted based on environmental characteristics to obtain an intermediate threshold. For example, when environmental characteristics degrade, the candidate threshold can be increased to obtain an intermediate threshold. When environmental characteristics do not degrade, the candidate threshold can be directly used as the intermediate threshold. Then, based on the linear velocity and angular velocity indicated by the real-time motion state, the intermediate threshold is adjusted using Formula 2 above to obtain the second anomaly threshold.

[0127] S101-4: For the interaction dimension, the control module is used to determine whether there is an anomaly in the interaction dimension based on whether an interaction control command is received; the interaction control command comes from the controller or the cloud.

[0128] Here, interactive control commands can be external commands initiated to seize control of the embodied intelligent terminal. These commands are unintended when the embodied intelligent terminal is in a fully autonomous navigation state (i.e., under the control of the processing module), typically corresponding to long-tail scenarios. Interactive control commands can usually originate from controllers, the cloud, or a host computer. For example, interactive control commands issued by controllers can include commands from a remote control or commands initiated when a controller presses the emergency stop button. Interactive control commands issued from the cloud are typically initiated by cloud technicians when they remotely monitor the embodied intelligent terminal and discover operational anomalies. Operational anomalies can include, but are not limited to, the discovery of operational hazards, system crashes of the embodied intelligent terminal, abnormal processing module algorithms, missed detections by manual detection, abnormal messages detected by the cloud-based high-computing-power diagnostic platform but no local alarm, and temporary emergencies (such as fire-activated emergency stops).

[0129] In the anomaly detection network, the corresponding monitoring method for the interaction dimension can be the interaction layer forced preemption monitoring method. This method is used to detect long-tail scenarios such as perception misses and manual intervention. The target data under the interaction dimension is the detected interaction control commands. For example, the interaction layer forced preemption monitoring targets the hidden long-tail fault scenarios where the local self-evaluation mechanism of the embodied intelligent terminal (i.e., the execution layer physical anomaly monitoring method and the spatiotemporal consistency soft divergence monitoring method) fails. Specifically, the highest priority interaction control commands from the cloud are triggered by sources including but not limited to: 1. Perception miss takeover based on human visual feedback, such as a remote dispatcher discovering a composite physical hazard that the embodied intelligent terminal's local physical obstacle classifier did not recognize based on high-dimensional visual semantics; 2. Asynchronous diagnostic takeover based on distributed digital twins, such as a cloud computing center determining, through in-depth mining of long-term telemetry data of the embodied intelligent terminal, that the embodied intelligent terminal's local advanced planning algorithm is in a nonlinear soft divergence state or that the main control hardware is at a critical point of collapse; 3. Collaborative security takeover based on global dynamic spatiotemporal fences, such as the forced change of operational strategy red lines due to sudden environmental events. Since the unexpected takeover of external interactive control commands is captured during fully autonomous navigation, which proves that there is an irreversible decision blind spot in the local cognitive loop, the control module can forcibly cut off the local closed loop and use the moment of seizure as the time anchor point to reversely extract the multidimensional shadow dataset in the double-ended circular buffer, thereby achieving accurate collection of long-tail scene data.

[0130] In practical implementation, the interaction layer in the control module can be used to forcibly preempt monitoring, real-time listening to the Control Authority Finite State Machine (CAFSM) in the embodied intelligent terminal. When the state of the CAFSM changes, it can be determined that an external interactive control command has been received. The control module identifies the specific interactive control command and determines whether there is an anomaly in the interaction dimension (i.e., an anomaly exists in the embodied intelligent terminal). The time of receiving the interactive control command is used as the detection time, and a scene label is determined based on the command content, such as a label indicating forced manual takeover or suspected missed detection. Then, the anomaly monitoring grid can immediately send an anomaly signal to the trigger manager so that the trigger manager can promptly detect the anomaly in the interaction dimension.

[0131] Thus, in engineering experience, such unexpected human intervention or cloud interaction usually occurs because on-site operators discover physical hazards that the sensing algorithm has completely missed (missed detection), necessitating manual intervention. Therefore, when control is forcibly seized in an instant, it can be directly defined as a major anomaly in the control loop, triggering data capture and thereby improving the accuracy and comprehensiveness of data capture.

[0132] Understandably, the trigger manager can monitor abnormal signals generated by various monitoring methods in the abnormal monitoring grid through polling. Once the trigger manager detects an abnormal signal, it records the time when the abnormal signal was obtained as the detection time, and can generate a corresponding scene label based on the detection dimension corresponding to the monitoring method that generated the abnormal signal. Then, based on the detection time and scene label, it generates a data acquisition command and sends it to the processing module.

[0133] In one embodiment, the first and second circular buffers have the same buffer depth, which indicates the historical duration corresponding to the historical data frames to be buffered. Here, the first circular buffer in the processing module and the second circular buffer in the control module have the same buffer depth. That is, the total length of the historical duration of the historical data frames to be buffered in the two circular buffers is the same. The historical duration corresponding to each historical data frame is the size of the time-domain window corresponding to each historical data frame, for example, the historical duration can be the most recent 15 seconds. The buffer depth can be dynamically adjusted according to the following steps: using the control module, the buffer depth is determined based on the real-time moving speed of the embody smart terminal and the environmental complexity of the current environment of the embody smart terminal.

[0134] Here, real-time movement speed is used to characterize the real-time speed of the embodied intelligent terminal. Environmental complexity is used to characterize the complexity of the environmental characteristics of the operating environment in which the embodied intelligent terminal operates. Typically, if the buffer depth of the two circular buffers is hard-coded to a fixed duration, a contradiction arises: "memory bandwidth and capacity surges during high-speed operation, while slow drift faults during low-speed operation cannot be traced." Therefore, an adaptive self-adjustment mechanism for buffer depth is needed. Similarly, a contradiction arises: "memory bandwidth and capacity surges during complex environments, while long-term recording during simple environments leads to severe information redundancy."

[0135] Therefore, to ensure the reasonable storage of data within the fixed-size dual-ended circular buffer, a control module can be used to determine the real-time movement speed of the embody intelligent terminal and the environmental complexity of its current environment. Then, based on the real-time movement speed and environmental complexity, the buffer depth of the dual-ended buffer can be dynamically adjusted synchronously.

[0136] For example, in scenarios where the real-time motion speed indicator of the embodied intelligent terminal is moving at high speed (e.g., real-time speed greater than 5 m / s) and the environmental complexity is low (e.g., low-dynamic pedestrian flow or fixed scenarios), the buffer depth T_max of the dual-end buffer is shortened to 3-5 seconds to reduce the rapid filling of the dual-end ring buffer by a large amount of sensor data during high-speed driving. Furthermore, due to sudden failures under high-speed conditions, the causal chain occurs very quickly, and the cause of the failure is generally apparent within 3 seconds before the accident occurs; therefore, the buffer depth T_max needs to be extended to 15 seconds. Conversely, when the real-time motion speed indicator of the embodied intelligent terminal is inspecting at low speed (e.g., real-time speed not greater than 5 m / s) and the environmental complexity is high (e.g., high-dynamic pedestrian flow scenarios), the buffer depth of the dual-end buffer is restored to 15 seconds.

[0137] Optionally, a polynomial mapping relationship with real-time movement speed and environmental complexity can be pre-set. Using the control module, the time domain window (i.e., buffer depth) of the double-ended circular buffer can be dynamically shrunk or expanded according to the polynomial mapping relationship and the real-time movement speed and environmental complexity. Under the premise of maintaining a constant total memory consumption of the double-ended circular buffer, high-density short-time domain capture under high-speed conditions and low-density long-time domain tracing under low-speed conditions can be achieved.

[0138] In one embodiment, S103 described above can be implemented according to the following steps: S103-1: Using the control module, determine the first historical data frame with the smallest difference between the data frame time and the detection time from the various historical data frames included in the second circular buffer.

[0139] Here, the historical data frames are the control data of each frame stored in the second circular buffer. Each historical time frame has a global timestamp synchronized by the gPTP protocol, which is the data frame time of the historical data frame. The first historical data frame is the historical data frame with the smallest difference between the data frame time and the detection time.

[0140] In practical implementation, when the anomaly monitoring grid in the control module detects an anomaly in any target detection dimension, the trigger manager can read the anomaly signal corresponding to that target detection dimension. Then, the circular buffer manager in the control module can be used to collect data from the local second circular buffer to gather cached data related to the anomaly occurring before and after the target detection dimension. During data collection, the circular buffer manager can first determine the first historical data frame with the smallest time difference from T_trigger, based on the anomaly detection time (T_trigger) corresponding to the target detection dimension and the data frame times of each historical data frame currently stored in the second circular buffer.

[0141] S103-2: Starting from the data frame time of the first historical data frame, backtrack for a first duration and append a second duration to obtain the first time window, and freeze the target cache data under the first duration backtracked from the data frame time of the first historical data frame.

[0142] Here, the target cache data refers to the control data within the first time period, traced back from the data frame time of the first historical data frame. This data belongs to the data prior to the occurrence of the anomaly corresponding to the target detection dimension. The reason for freezing the target cache data is to prevent the problem of excessive control data being cached in the second circular buffer after the anomaly occurs, which could overwrite the target cache data and make it uncollectible. At the same time, freezing only the target cache data and not the entire second circular buffer ensures that subsequent control data appended within the second time period can be written, thereby ensuring that the control data prior to the occurrence of the anomaly corresponding to the target detection dimension can be collected.

[0143] In practice, a circular buffer manager can be used to obtain the first time window before and after the anomaly corresponding to the target detection dimension, starting from the data frame time of the first historical data frame, and adding a first duration (T_pre) before and after the first duration (T_post) after the first duration. At the same time, the target cache data within the first duration of the forward backtracking is frozen to prevent the data before the anomaly from being overwritten.

[0144] S103-3: Starting from the memory address pointer of the first historical data frame in the second circular buffer, collect each second historical data frame located within the first time window, and determine the first cached data and unfreeze the target cached data according to the time range corresponding to each second historical data frame and the first time window.

[0145] Here, each second historical data frame constitutes the complete data before and after the occurrence of the anomaly corresponding to the target detection dimension. The time range corresponding to the first time window is used as the global timestamp of the first cached data.

[0146] In practical implementation, the circular buffer manager in the control module can be used to retrieve the second historical data frames within the first time window by using the memory address pointer of the first historical data frame in the second circular buffer as the acquisition anchor point. This process involves forward and backward pointer retrieval to obtain the first historical data frames. These second historical data frames are then combined to form the first cached data, and the time range corresponding to the first time window is used as the global timestamp for each first cached data frame. Afterward, the freeze on the target cached data in the second circular buffer can be lifted to allow for the storage of subsequent new control data.

[0147] It can be understood that the step of using the processing module to determine the second time window according to the detection time and collecting the second buffer data within the second time window from the first circular buffer is similar to the step and principle of using the control module to collect the first buffer data, and will not be repeated here.

[0148] For example, when the multivariate anomaly monitoring network in the control module detects a fault trigger and the trigger time is T_trigger (that is, the multivariate anomaly monitoring network detects that there is an abnormality in the target detection dimension and the abnormality detection time is T_trigger), the trigger manager can issue a "data acquisition instruction" to the processing module. However, there may be real-time delay (τ_delay) caused by Ethernet or PCIe transmission in the communication between the processing module and the control module, so when the processing module receives the data acquisition instruction, the time has reached T_trigger+τ_delay. To ensure that the processing module and the control module intercept buffer data under the same time window, the following closed-loop logic can be implemented: After the control module detects the fault trigger, the "data acquisition instruction" sent to the processing module must include the absolute trigger timestamp T_trigger. The processing module responds in a delayed manner and performs asynchronous backtracking. Specifically, the write pointer of the first circular buffer of the processing module does not need to stop immediately at the moment of receiving the data acquisition instruction. Because the first circular buffer itself has the capability of historical temporary storage, even after τ_delay milliseconds, as long as τ_delay<<T_max (that is, the buffer depth), the historical perception data before the fault occurrence will not be overwritten (the transmission delay is usually several milliseconds, and the minimum T_max of the buffer is 3~5ms, so it can ensure that the historical perception data will not be overwritten). Then, reverse retrieval and slicing of double-ended pointers can be performed in the double-ended circular buffer to collect the buffer interval in the double-ended circular buffer. For example, after receiving the instruction with T_trigger, the respective circular buffer managers of the processing module and the control module start independent pointer retrieval programs respectively. For each circular buffer manager, it can reversely traverse the timestamp metadata of each historical data frame in the corresponding circular buffer to find the memory address pointer corresponding to the data frame closest to T_trigger. Taking this pointer as the core anchor, the two ends respectively search T_pre (history before fault) forward and T_post (evolution after fault) backward, and accurately extract the corresponding memory slice to obtain double-ended data slices. The global timestamp intervals of the extracted double-ended data slices are completely closed and aligned (for example, the processing module and the control module fully retain the buffer data from 12:00:00 to 12:00:15 on June 1, 2026). Finally, the two segments of buffer data with perfectly aligned global timestamps are marked with the same "fault label" and uploaded to the cloud, thereby completely eliminating the data dislocation caused by inter-chip communication delay.

[0149] Thus, the shadow data reverse interception mechanism based on a double-ended circular buffer provided in this application utilizes a FIFO circular buffer that is permanently located at both ends of the brain and cerebellum. After an abnormality is triggered, it can achieve historical data loop-back interception and timely traceability of subsequent data by freezing memory, backtracking forward, and continuously recording backward. This ensures the integrity and comprehensiveness of the long-tail data collected when long-tail scenarios occur.

[0150] like Figure 2The diagram shown illustrates the terminal architecture interaction of a data acquisition method provided in this application embodiment. The embodied intelligent terminal may include a processing module as the "brain" and a control module as the "cerebellum." The "brain" may include a perception and scene recognition algorithm based on multiple cameras and LiDAR, a SLAM mapping and localization algorithm, and a first circular buffer (ring_buffer1). The perception and scene recognition algorithm implements deep learning perception services based on environmental data and sends the perception results to the SLAM mapping and localization algorithm and caches them in the first circular buffer. The SLAM mapping and localization algorithm, based on the perception results, implements large-scene SLAM mapping services, global path planning services, etc., and sends information such as global planned pose and global movement trajectory to the control closed-loop core of the cerebellum. The first circular buffer caches environmental perception data for each frame (e.g., original image + point cloud data). The cerebellum includes a proprioception node (e.g., IMU + high-frequency wheel velocities / foot odometer), a control closed-loop core, a multi-dimensional anomaly monitoring grid, a trigger manager, and a second circular buffer (ring_buffer2). The system comprises three main components: a proprioception node for collecting state data of the embodied intelligent terminal during real-time operation, such as IMU data and wheel speed; a control closed-loop core for running Model Predictive Control (MPC) and motion control algorithms, providing real-time control of the embodied intelligent terminal based on global planning pose and global movement trajectory information output from the control unit; a multi-dimensional anomaly monitoring grid with execution-layer physical anomaly monitoring, spatiotemporal consistency soft divergence monitoring, and interaction-layer forced preemption monitoring to detect anomalies across various dimensions and issue anomaly information upon detection; a second circular buffer for caching various control data (such as state data and control commands); and a trigger manager for monitoring anomaly signals in the multi-dimensional anomaly monitoring grid. Upon detection of an anomaly signal, the trigger manager can send a data acquisition command (which can be a high-priority interrupt command) to the control unit, triggering the acquisition of the corresponding second-buffered data in the first circular buffer. Simultaneously, upon detection of an anomaly signal, the trigger manager can also generate a local snapshot command to trigger the acquisition of the corresponding second-buffered data in the local first circular buffer. After the cerebral and cerebellar ends collect cached data related to the abnormality in their respective circular buffers, they can package the data and asynchronously upload it to the cloud for storage.

[0151] Thus, this application departs from the traditional approach of data capture dominated by the "sensory end / brain end," defining the high-real-time control closed-loop end (cerebellum end) as the arbitration center for global anomalies. Leveraging the unique position of the cerebellum end as the data intersection point of "physical execution" and "high-level algorithms on the brain end," a three-in-one anomaly monitoring grid is constructed, encompassing "physical hard anomalies," "spatiotemporal soft divergence," and "human-machine strong preemption." By combining the corresponding ring buffers of the brain end and the cerebellum segment, high-precision, reverse-triggered capture with historical backtracking is achieved for long-tail scenario data.

[0152] In one embodiment, S105 described above can be implemented according to the following steps: S105-1: Using the control module, based on the detection time and scene label, divide the first cache data into compressed cache data and uncompressed cache data.

[0153] Here, different scene labels correspond to different data stream lengths that can be compressed. For example, the data stream length that can be compressed is shorter for execution layer physical overload / physical collision / slippage labels or manual forced takeover / suspected missed detection labels, while the data stream length that can be compressed is longer for spatiotemporal disconnect labels / SLAM positioning divergence labels / high tracking error labels.

[0154] The cached data to be compressed is the data in the first cached data that can be compressed. The uncompressed cached data is the data in the first cached data other than the cached data to be compressed. This data is not allowed to be compressed and needs to be uploaded to the cloud without loss.

[0155] In practical implementation, the control module can determine the compressible data stream length based on the scene label corresponding to the first cached data, and determine the data to be compressed that matches the cluster length based on the detection time. Data other than compressed data in the first cached data is then treated as uncompressed cached data. For example, if the scene label is an execution layer physical overload label, the compressible data stream length can be determined to be the data stream before the first target time corresponding to the detection time and after the second target time corresponding to the detection time. The first target duration can be 3 seconds, and the second target duration can also be 3 seconds. That is, for the first cached data, data within 3 seconds before the data frame time corresponding to the detection time and data within 3 seconds after the data frame time corresponding to the detection time can be treated as uncompressed cached data, while other data is treated as data to be compressed. This is because for core timing data streams, high-frequency, lightweight, and absolutely lossless compression is required. Therefore, data objects that cannot be compressed (i.e., uncompressed cached data) can include: high-frequency IMU raw data 1-3 seconds before triggering (and 1-3 seconds after triggering), chassis motor current, CAN bus control commands, cerebellum state machine trajectory increments, etc. Buffered data of each frame whose difference between the data frame time and the data frame time corresponding to the detection time is greater than 3 seconds are used as data to be compressed.

[0156] S105-2: Compress the data to be compressed in the cache to obtain the first compressed data, and merge the first compressed data and the uncompressed cache data into the first target data according to the time sequence.

[0157] In practice, any data compression algorithm can be used to compress the cached data to be compressed, resulting in first compressed data. Then, according to the data frame time corresponding to the first compressed data in the first cached data, the first compressed data and uncompressed cached data can be merged into first target data arranged in chronological order.

[0158] S105-3: Using the processing module, determine the data compression method for the second cached data based on the scene label, and use the data compression method to compress the second cached data to obtain the second target data.

[0159] Here, different scene labels correspond to different data compression methods, which indicate how environmental perception data is compressed. For example, the data compression method corresponding to the execution layer physical overload / physical collision / slippage label or manual forced takeover / suspected missed detection can be full data compression, while the data compression method corresponding to the spatiotemporal disjoint label / SLAM localization divergence label / high tracking error label can be region of interest (ROI) extraction.

[0160] In practice, the processing module can be used to determine the target data compression method for the second cached data based on the scene label corresponding to the second cached data. Then, the environmental awareness data of each frame in the second cached data can be compressed according to the method to obtain the compressed second target data.

[0161] For example, the environmental awareness data (including camera images / video streams and raw point cloud data) of each frame in the second cached data pair can be subjected to redundant lossy compression or ROI extraction to obtain the second target data.

[0162] S105-4: Transmit the first target data and the second target data to cloud storage.

[0163] In practice, the control module can be used to transmit the first target data to the cloud, and the processing module can be used to transmit the second target data to the cloud.

[0164] Thus, the automated labeling and cloud-based closed-loop method based on anomaly sources provided in this application automatically injects scene labels according to the anomaly type corresponding to the detection dimension when the anomaly occurs, and asynchronously transmits the shadow snapshot carrying the scene labels back to the cloud, effectively completing the data closed loop.

[0165] In another embodiment, this application also provides a method for dynamically adjusting upload priority based on network conditions. Specifically, for the above S105-4, it can be implemented according to the following steps: S105-4-1: Based on the scene tag, determine the upload priority of the first target data and the second target data, and store the first target data and the second target data carrying the upload priority into the preset upload queue.

[0166] Here, the preset upload queue can be a data transmission queue set up in the embodied smart terminal. This queue can be managed using a preset queue manager based on data value metrics. The preset upload queue can be used to store target data uploaded from the processing module and the control module in each iteration. A target data item can include a first target data item and a second target data item, and different target data items can be the first target data item and the second target data item uploaded in different iterations.

[0167] Upload priority is used to characterize the importance of target data. The more important the target data, the more promptly it needs to be uploaded to the cloud. Therefore, different upload priorities can correspond to different upload methods. Upload methods may include, for example, immediate streaming using 5G / 4G networks, or delayed streaming using bandwidth-limited or Wi-Fi networks.

[0168] In practical implementation, the control module can determine the upload priority of the first target data based on the scene label corresponding to it. For example, the upload priority is highest for tags such as execution layer physical overload / physical collision / slippage or suspected manual takeover / perception miss detection, while the upload priority is lowest for tags such as spatiotemporal discontinuity / SLAM positioning divergence / high tracking error. Similarly, the processing module can determine the upload priority of the second target data based on the scene label corresponding to it. Understandably, for the first and second target data under the same anomaly, their corresponding scene labels and upload priorities are the same. Then, the control module can upload the first target data carrying the upload priority to a preset upload queue, and the processing module can upload the second target data carrying the upload priority to the preset upload queue.

[0169] Optionally, the control module and processing module can be used to transmit the first target data and the second target data to preset upload queues respectively. Then, the queue manager corresponding to the preset upload queues can be used to determine the corresponding upload priorities based on the scene tags corresponding to the first target data and the second target data.

[0170] S105-4-2: Upload each target data to the cloud according to the upload priority and upload method corresponding to each target data in the preset upload queue.

[0171] Here, a target data is the first target data and the second target data of a certain upload stored in the preset upload queue. Since the preset upload queue allows the target data to be transmitted with delay, the preset upload queue can store the first and second target data uploaded by the same smart terminal in different uploads, or it can store the first and second target data uploaded by different smart terminals in different uploads.

[0172] In practice, a queue manager can be used to upload each target data stored in the preset upload queue to the cloud for storage according to the upload priority and the corresponding upload method.

[0173] For example, the first and second target data under tags such as execution layer physical overload / physical collision / manual forced takeover are high-value, real-time data, so they have the highest transmission priority and can be immediately streamed to the cloud using 5G / 4G networks. The first and second target data under SLAM positioning divergence tags / high tracking error tags are medium-to-low value data, so their transmission priority is relatively low. They can be streamed using bandwidth-limited or wireless network-based delayed streaming methods, and uploaded to the cloud when network conditions permit.

[0174] like Figure 3The diagram shown is another flowchart of a data acquisition method provided in an embodiment of this application, which may include the following steps: If an anomaly is detected in any target detection dimension by the multivariate anomaly monitoring grid, the cerebellum sends an interrupt command. This interrupt command can include a data acquisition command sent to the cerebral end and a local snapshot command for the cerebellum itself. For example, the trigger manager can send a "data acquisition signal" to the cerebral end via a high-priority interrupt bus across the chip (such as a PCIe interrupt line or a dedicated level pin) while freezing the local circular buffer of the cerebellum.

[0175] The cerebral / cerebellar end can respond to an interrupt command by freezing the memory of its circular buffer. Specifically, this freezing can involve freezing the target cached data for the first time interval. Then, the cerebral / cerebellar end can extract cached data from the circular buffer for the first time interval (T_pre) and continue recording subsequent data for the second time interval (T_post), using this data as cached data related to the anomaly. For example, the cerebral / cerebellar end can stop the FIFO overwrite mechanism of the circular buffer, locking and copying the historical static memory block whose anomaly trigger time (T_trigger) is pushed back by T_pre seconds (e.g., 5 seconds). Simultaneously, the cerebral / cerebellar end can maintain the current sensor input and continue appending and copying the real-time data stream for T_post seconds (e.g., 10 seconds).

[0176] Then, the trigger manager can determine the scene labels of the collected data based on the anomaly type corresponding to the target detection dimension, and upload the data carrying the scene labels to the cloud. For example, the trigger manager can perform automatic label injection and asynchronous return steps, and automatically generate long-tail scene labels based on the specific anomaly source that caused the anomaly. If triggered by the execution layer physical anomaly monitoring method, the [execution layer physical overload / physical collision] tag will be injected.

[0177] If triggered by the spatiotemporal consistency soft divergence monitoring method, the [spatiotemporal disconnect / SLAM divergence / high tracking error] tag will be injected.

[0178] If the monitoring is forcibly taken over by the interaction layer, the tag "[Manual forced takeover / Suspected missed detection]" will be injected.

[0179] In this way, it is possible to retrieve images, point clouds, pose trajectories, and control cost (i.e., ...) from the previous 5 seconds and the next 10 seconds (a total of 15 seconds) when an anomaly is triggered. The curve and scene label are packaged into a shadow snapshot, stored in a local low-priority preset upload queue, and then asynchronously uploaded to the cloud when the network is idle (such as when connecting to factory Wi-Fi or charging at night).

[0180] Optionally, this application embodiment also provides a method for correlation analysis and causal inference of anomaly sources. Specifically, after receiving the first and second target data from the embodied intelligent terminal in the cloud, the system can mine the spatiotemporal correlation of anomalies corresponding to different target data based on the global timestamps and scene tags of the first and second target data. Based on the correlation results, richer hierarchical tags are generated for the target data, along with anomaly root cause analysis. This allows algorithm engineers in the cloud to improve the speed of anomaly root cause analysis based on the anomaly causal chain indicated by the hierarchical tags. For example, when multiple anomalies occur spatiotemporally correlated (such as a spatiotemporal consistency residual occurring first, followed by a physical collision 200ms later), the cloud can automatically construct anomaly causal chain and generate structured metadata (such as SLAM drift → tracking error surge → physical collision). Based on this structured metadata, richer hierarchical tags are injected into each target data.

[0181] In one embodiment, when a logic chip is provided, the logic chip is used to obtain a first anomaly threshold in the physical bus dimension, and to obtain the motor current and the absolute acceleration output by the inertial measurement unit in multiple consecutive cycles; based on the motor current and the absolute acceleration, it is determined whether an anomaly exists in the physical bus dimension.

[0182] Here, the logic chip can be a pre-configured field programmable gate array (FPGA) chip, which can be used to implement the above-mentioned physical anomaly monitoring method at the execution layer.

[0183] In practical implementation, an FPGA can be deployed on the intelligent terminal, in addition to the control module and processing module. This FPGA chip can execute the aforementioned execution-layer physical anomaly monitoring method in real time to detect whether there are any anomalies at the physical bus dimension. If an anomaly is found, an anomaly signal is generated and sent to the trigger manager to trigger the collection of cached data corresponding to the anomaly. The step of using the FPGA chip to execute the aforementioned execution-layer physical anomaly monitoring method to determine whether there are any anomalies at the physical bus dimension simply involves switching the execution entity from the control module to the FPGA chip; the specific execution method remains unchanged. Therefore, the anomaly detection process in the FPGA chip can be referred to the descriptions in S101-1 and S101-2 above, and will not be repeated here.

[0184] In this way, by adding FPGAs outside the brain and cerebellum, hardware-level monitoring of motor current, IMU signals, etc. can be achieved, thereby triggering the freezing of the ring buffer at the millisecond level. This achieves pure hardware-based anomaly capture without software dependence, improving the reliability of data acquisition to the component level.

[0185] In one embodiment, to improve the comprehensiveness of data collection during clustered operations on intelligent terminals, this application also provides a remote triggering capture method based on cloud-edge collaboration. Specifically, collaborative data capture can be achieved according to the following steps C1~C3: C1: Receives data collection notifications from the cloud; the data collection notification is initiated by the cloud after receiving the data collection instruction uploaded by the target terminal; the target terminal is any other embodied smart terminal in the terminal cluster where the embodied smart terminal is located.

[0186] Here, the terminal cluster can be a cluster composed of multiple identical embodied intelligent terminals, used to implement cluster operations. Each embodied intelligent terminal can run the data acquisition method described in the above embodiments. The target terminal can be any embodied intelligent terminal in the terminal cluster. When the target terminal detects an anomaly in any target detection dimension using the above-described multivariate anomaly detection grid, it can send a data acquisition command including scene labels and detection time to the cloud.

[0187] Data collection notifications can be data capture notifications that are synchronously sent from the cloud to each embodied smart terminal in the terminal cluster. These notifications can be initiated by the cloud after receiving a data collection instruction uploaded by any target terminal in the embodied smart terminal cluster.

[0188] For example, when the embodied intelligent terminals in the terminal cluster are performing cluster operations, if any embodied intelligent terminal's multivariate anomaly detection grid detects an anomaly in any target detection dimension, that embodied intelligent terminal can act as the target terminal and generate a data acquisition command, which is then sent to the cloud. The cloud can receive this data acquisition command and, based on the detection time and anomaly label carried in the command, synchronously send data acquisition notifications to each embodied intelligent terminal in the terminal cluster. Then, each embodied intelligent terminal in the terminal cluster can receive the data acquisition notification.

[0189] C2: Using the controlled module, collect the third cache data from the second ring buffer, which is located within the third time window corresponding to the collection time in the data collection notification; and using the processing module, collect the fourth cache data from the first ring buffer, which is located within the fourth time window corresponding to the collection time.

[0190] Here, the acquisition time is the detection time carried in the data acquisition command, which is the time when the target terminal detects an anomaly in the target detection dimension (T_trigger). With time synchronization between the processing module and control module in the embodied intelligent terminal, the time range corresponding to the three time windows is consistent with the time range corresponding to the fourth time window. The process of determining the third and fourth time windows using the acquisition time can be referred to the description of determining the first and second time windows in S103 and S104 above, and will not be repeated here. The third cached data is the cached data collected from the second circular buffer, corresponding to the target detection dimension before and after the anomaly. The fourth cached data is the cached data collected from the first circular buffer, corresponding to the target detection dimension before and after the anomaly.

[0191] For example, for any intelligent terminal in the terminal cluster, after receiving a data collection notification, the control module can parse the collection time and scene tag from the data collection notification. Then, following the steps S102 to S104 above, the third cached data and the fourth cached data are collected.

[0192] C3: Transmit the third cache data, the fourth cache data, and the scene tags in the data collection notification to cloud storage.

[0193] For example, for any intelligent terminal in the terminal cluster, after the terminal collects the third cache data and the fourth cache data, the third cache data, the fourth cache data, and the scene tags parsed from the data collection notification can be asynchronously uploaded to the cloud for storage. The specific implementation process of C3 mentioned above can be found in the description of S105 above, and will not be repeated here.

[0194] In this way, when the embodied intelligent terminal detects an anomaly in any target detection dimension, it can send a data collection command to the cloud to enable remote batch management of long-tail data collection from multiple embodied intelligent terminals, adapting to clustered operation scenarios.

[0195] In one embodiment, for the step of "obtaining the first abnormal threshold in the physical bus dimension" in S101-1 above, in addition to using the step of obtaining it based on the threshold calibration table and online feedforward correction method described above, an adaptive threshold determination method based on lightweight edge inference deployed locally in the control module can also be adopted. Specifically, the first abnormal threshold can be determined by using the prediction model in the control module according to the current adhesion coefficient of the driving surface and the operating conditions of the embodied intelligent terminal.

[0196] Here, the prediction model is a lightweight edge inference network model deployed locally by the control module. This model is used to dynamically infer the first anomaly threshold. Operating conditions may include, but are not limited to, the working mode of the embedded smart terminal, motor attributes, power information, environmental characteristics, etc.

[0197] For example, a control module can be used to determine the current adhesion coefficient of the road surface on which the embodied intelligent terminal is traveling, as well as the current operating condition of the embodied intelligent terminal, based on the terminal's motion state. The current adhesion coefficient and the current operating condition can then be input into a prediction model to obtain a first anomaly threshold.

[0198] In this way, by deploying a lightweight anomaly inference model in the control module to adaptively adjust the anomaly threshold, the problem of false triggering due to fixed thresholds can be effectively avoided, the accuracy of data capture can be improved, and it can be adapted to complex and ever-changing operating environments.

[0199] In summary, the data acquisition method provided in this application can achieve "precise dehydration" of long-tail data in long-tail scenarios, reducing the data acquisition rate from 100% (full recording in existing technologies) to less than 0.5% triggered only in abnormal long-tail scenarios. This saves over 99% of cloud storage and bandwidth costs without losing any high-value long-tail scenario data, significantly improving the overall efficiency of data mining. Furthermore, by "reverse-defining anomalies by the control module," the blind spots in the processing module's perception are completely eliminated, changing the logic of "data recording triggered by the processing module itself." Even if the processing module's perception algorithm completely fails, as long as there are manual interaction commands or physical collisions to the chassis, the control module can forcibly trigger the capture of long-tail scenario data through "result-based reverse engineering." Furthermore, this application possesses the long-tail tracing capability to "travel through the past." Relying on the circular buffer that is permanently stationed at both ends of the processing module and the control module, it can achieve the capture of not only the damaged fragments after the fault, but also "panoramic high-definition reconnaissance data" containing the original seed period context within the first time period before the fault occurred. This provides the most complete long-tail data for sporadic divergence in SLAM positioning and instantaneous missed detection in perception, greatly accelerating the iterative evolution of core algorithms for autonomous driving and robotics.

[0200] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0201] Furthermore, based on the same inventive concept, embodiments of this disclosure also provide a robot, such as... Figure 4The diagram shown is a schematic representation of the architecture of a robot according to an embodiment of this application. The robot 40 includes a processing module 401 for environmental perception and task planning, a control module 402 for real-time motion control, and a transmission module 403 for data transmission. The processing module 401 is provided with a first circular buffer for caching environmental perception data for each frame; the control module 402 is provided with a second circular buffer for caching control data for each frame. The control module 402 is used to perform anomaly detection on the robot based on target data under multiple detection dimensions. The multiple detection dimensions include physical bus dimension, processing module output dimension, and interaction dimension, and different detection dimensions correspond to different anomaly scenarios. If an anomaly is detected in any target detection dimension, a data acquisition instruction is generated according to the detection time and sent to the processing module 401. First cached data located within the first time window corresponding to the detection time is acquired from the second circular buffer. The first time window is obtained by backtracking a first duration and appending a second duration, starting from the data frame time corresponding to the detection time. The processing module 401 is configured to respond to the data acquisition signal, determine a second time window based on the detection time, and acquire second cached data located within the second time window from the first annular buffer. The transmission module 403 is used to transmit the first cached data, the second cached data, and the scene labels corresponding to the target detection dimension to cloud storage.

[0202] For the specific implementation steps of the processing module 401, control module 402 and transmission module 403 in the robot mentioned above, please refer to the introduction of the data acquisition method above, which will not be repeated here.

[0203] Based on the same inventive concept, this application also provides a data acquisition device for a avatar smart terminal corresponding to the data acquisition method of the avatar smart terminal. Since the principle of the device in this application is similar to the data acquisition method of the avatar smart terminal described above in this application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0204] like Figure 5 The diagram shown is a schematic of a data acquisition device for a cloaked smart terminal according to an embodiment of this application. The cloaked smart terminal includes a processing module for environmental perception and task planning and a control module for real-time motion control. The processing module is provided with a first circular buffer for caching environmental perception data of each frame; the control module is provided with a second circular buffer for caching control data of each frame. The device includes: The detection unit 501 is used to perform anomaly detection on the embodied smart terminal based on target data under multiple detection dimensions using the control module; the multiple detection dimensions include physical bus dimension, processing module output dimension and interaction dimension, and different detection dimensions correspond to different anomaly scenarios; The generation unit 502 is used to generate a data acquisition instruction based on the detection time if any abnormality is detected in any target detection dimension, and send it to the processing module. The first acquisition unit 503 is used to acquire first cached data located within the first time window corresponding to the detection time from the second annular buffer using the control module; the first time window is obtained by backtracking a first duration and appending a second duration, respectively, starting from the data frame time corresponding to the detection time. The second acquisition unit 504 is used to utilize the processing module in response to the data acquisition command, determine a second time window based on the detection time, and acquire second cached data located within the second time window from the first annular buffer. The transmission unit 505 is used to transmit the first cached data, the second cached data, and the scene labels corresponding to the target detection dimension to cloud storage.

[0205] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0206] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0207] Based on the same technical concept, embodiments of this application also provide a computer device. (Refer to...) Figure 6 The diagram shown is a structural schematic of a computer device provided in an embodiment of this application, comprising: The processor 601, memory 602, and bus 603 are included. Memory 602 stores machine-readable instructions executable by the processor 601. The processor 601 executes the machine-readable instructions stored in memory 602. When the machine-readable instructions are executed by the processor 601, the processor 601 executes the above-described S101 to S105.

[0208] The aforementioned memory 602 includes a main memory 6021 and an external memory 6022. The main memory 6021, also known as internal memory, is used to temporarily store the computational data in the processor 601, as well as the data exchanged with external memory such as a hard disk 6022. The processor 601 exchanges data with the external memory 6022 through the main memory 6021. When the computer device is running, the processor 601 and the memory 602 communicate through the bus 603, so that the processor 601 executes the execution instructions mentioned in the above method embodiments.

[0209] This application also provides a computer-readable storage medium storing a computer program. When a processor executes the computer program, it performs the steps of the data acquisition method for a personalized smart terminal described in the above-described method embodiments. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0210] This application also provides a computer program product, which carries program code. The instructions included in the program code can be used to execute the steps of the data acquisition method of the smart terminal described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0211] The computer program product can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0212] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0213] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0214] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0215] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0216] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the user through pop-up information or by asking the user to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0217] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A data acquisition method for a personal intelligent terminal, characterized in that, The embodied intelligent terminal includes a processing module for environmental perception and task planning and a control module for real-time motion control; the processing module is provided with a first circular buffer for caching environmental perception data of each frame; The control module is equipped with a second circular buffer for caching control data for each frame; the method includes: The control module performs anomaly detection on the embodied intelligent terminal based on target data across multiple detection dimensions. These multiple detection dimensions include a physical bus dimension, a processing module output dimension, and an interaction dimension, with different detection dimensions corresponding to different anomaly scenarios. The target data under the physical bus dimension includes the motor current and the absolute acceleration output by the inertial measurement unit over multiple consecutive cycles. The target data under the processing module output dimension includes the target's movement trajectory and the positioning increment within a preset time window. The target data under the interaction dimension includes whether an interactive control command has been received. If any target detection dimension is detected as abnormal, a data acquisition instruction is generated based on the detection time and sent to the processing module. Using the control module, first cached data located within the first time window corresponding to the detection time is collected from the second circular buffer; the first time window is obtained by backtracking a first duration and appending a second duration, respectively, starting from the data frame time corresponding to the detection time; The processing module responds to the data acquisition command, determines a second time window based on the detection time, and acquires second cached data located within the second time window from the first annular buffer. The first cached data, the second cached data, and the scene labels corresponding to the target detection dimension are transmitted to cloud storage.

2. The method according to claim 1, characterized in that, The step of using the control module to perform anomaly detection on the embodied smart terminal based on target data across multiple detection dimensions includes: Regarding the physical bus dimension, in the absence of any logic chip related to the physical bus dimension, the control module is used to obtain the first abnormal threshold under the physical bus dimension, as well as the motor current and the absolute acceleration output by the inertial measurement unit in multiple consecutive cycles. Based on the motor current, the absolute acceleration, and the first anomaly threshold, determine whether an anomaly exists in the physical bus dimension; Regarding the output dimension of the processing module, the control module uses the target movement trajectory output by the processing module and the positioning increment within a preset time window to determine whether there is an anomaly in the output dimension of the processing module. Regarding the interaction dimension, the control module determines whether an anomaly exists in the interaction dimension based on whether an interaction control command is received; the interaction control command comes from the controller or the cloud.

3. The method according to claim 2, characterized in that, The first abnormal threshold includes a current abnormal threshold and an acceleration abnormal threshold; The step of determining whether an anomaly exists in the physical bus dimension based on the motor current, the absolute acceleration, and the first anomaly threshold includes: Using the control module, the current mutation rate of the motor current and the acceleration mutation rate of the absolute acceleration under the consecutive multiple cycles are determined; If the current mutation rate is greater than the current anomaly threshold in multiple consecutive cycles, and / or the acceleration mutation rate is greater than the acceleration anomaly threshold in multiple consecutive cycles, an anomaly is determined to exist in the physical bus dimension.

4. The method according to claim 2, characterized in that, The step of obtaining the first anomaly threshold under the physical bus dimension includes: Using the control module, the current adhesion coefficient of the road surface on which the smart terminal is traveling is determined based on the motion state of the smart terminal. From a pre-set threshold calibration table, an abnormal threshold to be used is obtained based on the current adhesion coefficient, the working mode of the embodied smart terminal, and the motor attributes; the threshold calibration table carries the correlation between various abnormal thresholds and the adhesion coefficient, working mode, and motor attributes; If the current adhesion coefficient is different from the previously determined historical adhesion coefficient, the anomaly threshold to be used is adjusted according to the current adhesion coefficient and the historical adhesion coefficient to obtain the first anomaly threshold.

5. The method according to claim 2, characterized in that, The control module, based on the target movement trajectory output by the processing module and the positioning increment within a preset time window, determines whether there are any anomalies in the dimension output by the processing module, including: Using the control module, a second abnormal threshold is obtained in the output dimension of the processing module, and the trajectory error of the embodied intelligent terminal is determined based on the real-time motion trajectory of the embodied intelligent terminal and the target movement trajectory. Based on the target moving speed in the embodied intelligent terminal within the preset time window, determine the target displacement increment of the embodied intelligent terminal within the preset time window; The positioning error of the embodied intelligent terminal is determined based on the target displacement increment and the positioning increment. If the trajectory error is greater than a preset error, and / or the positioning error is greater than a second anomaly threshold for multiple consecutive cycles, an anomaly is determined to exist in the output dimension of the processing module.

6. The method according to claim 5, characterized in that, The step of using the control module to obtain the second anomaly threshold in the output dimension of the processing module includes: Using the control module, a candidate threshold pre-calibrated for the positioning error is obtained based on the movement mode of the smart terminal. Based on the motion state of the avatar smart terminal and the environmental characteristics of the environment in which the avatar smart terminal is currently located, the candidate threshold is adjusted to obtain the second abnormal threshold.

7. The method according to claim 1, characterized in that, The first circular buffer and the second circular buffer have the same buffer depth, and the buffer depth is used to indicate the historical duration corresponding to the historical data frames that the buffer needs to cache; The buffer depth is dynamically adjusted according to the following steps: Using the control module, the depth of the buffer is determined based on the real-time moving speed of the avatar smart terminal and the environmental complexity of the environment in which the avatar smart terminal is currently located.

8. The method according to claim 1, characterized in that, The step of using the control module to collect first cache data located within the first time window corresponding to the detection time from the second annular buffer includes: Using the control module, the first historical data frame with the smallest difference between the data frame time and the detection time is determined from the various historical data frames included in the second circular buffer. Starting from the data frame time of the first historical data frame, the first time window is obtained by tracing back a first duration and appending a second duration, and the target cached data under the first duration traversed back from the data frame time of the first historical data frame is frozen. Starting from the memory address pointer of the first historical data frame in the second circular buffer, collect each second historical data frame located within the first time window, and determine the first cached data and unfreeze the target cached data according to the time range corresponding to each second historical data frame and the first time window.

9. The method according to claim 1, characterized in that, The step of transmitting the first cached data, the second cached data, and the scene labels corresponding to the target detection dimension to cloud storage includes: Using the control module, based on the detection time and the scene label, the first cached data is divided into compressed cached data and uncompressed cached data; The cached data to be compressed is compressed to obtain first compressed data, and the first compressed data and the uncompressed cached data are merged into first target data according to the time sequence. Using the processing module, a data compression method for the second cached data is determined based on the scene label, and the second cached data is compressed using the data compression method to obtain the second target data; The first target data and the second target data are transmitted to cloud storage.

10. The method according to claim 9, characterized in that, The first target data and the second target data are transferred to cloud storage, including: Based on the scene tag, determine the upload priority of the first target data and the second target data, and store the first target data and the second target data carrying the upload priority into a preset upload queue; Each target data is uploaded to the cloud according to the upload priority and corresponding upload method in the preset upload queue.

11. The method according to claim 2, characterized in that, The method further includes: With the logic chip in place, the logic chip is used to obtain the first abnormal threshold in the physical bus dimension, and to obtain the motor current and the absolute acceleration output by the inertial measurement unit in multiple consecutive cycles. Based on the motor current and the absolute acceleration, determine whether there is an anomaly in the physical bus dimension.

12. The method according to claim 1, characterized in that, The processing module and the control module are different chips; or, the processing module and the control module are different functional domains divided from the same chip.

13. The method according to claim 1, characterized in that, The method further includes: Receive data collection notifications from the cloud; the data collection notifications are initiated by the cloud after receiving data collection instructions uploaded by the target terminal; the target terminal is any other embodied intelligent terminal in the terminal cluster where the embodied intelligent terminal is located; Using the control module, third cache data within a third time window corresponding to the acquisition time in the data acquisition notification is collected from the second annular buffer; and using the processing module, fourth cache data within a fourth time window corresponding to the acquisition time is collected from the first annular buffer. The third cached data, the fourth cached data, and the scene tags in the data collection notification are transmitted to cloud storage.

14. The method according to claim 2, characterized in that, The step of obtaining the first anomaly threshold under the physical bus dimension includes: Using the prediction model in the control module, a first anomaly threshold is determined based on the current adhesion coefficient of the road surface and the operating conditions of the intelligent terminal.

15. A robot, characterized in that, The robot is used to execute the data acquisition method of the embodied intelligent terminal as described in any one of claims 1 to 14; the robot includes a processing module for environmental perception and task planning, a control module for real-time motion control, and a transmission module for data transmission; the processing module is provided with a first circular buffer for caching each frame of environmental perception data; the control module is provided with a second circular buffer for caching each frame of control data; wherein: The control module is used to perform anomaly detection on the robot based on target data under multiple detection dimensions. These multiple detection dimensions include a physical bus dimension, a processing module output dimension, and an interaction dimension, with different detection dimensions corresponding to different anomaly scenarios. The target data under the physical bus dimension includes motor current and absolute acceleration output by the inertial measurement unit over multiple consecutive cycles. The target data under the processing module output dimension includes the target movement trajectory and the positioning increment within a preset time window. The target data under the interaction dimension includes whether an interaction control command has been received. If an anomaly is detected in any target detection dimension, a data acquisition command is generated based on the detection time and sent to the processing module. First buffered data located within a first time window corresponding to the detection time is acquired from the second annular buffer. The first time window is obtained by retrospectively counting backward for a first duration and appending a second duration, starting from the data frame time corresponding to the detection time. The processing module is configured to respond to the data acquisition signal, determine a second time window based on the detection time, and acquire second cached data located within the second time window from the first annular buffer. The transmission module is used to transmit the first cached data, the second cached data, and the scene labels corresponding to the target detection dimension to cloud storage.

16. A data acquisition device for a smart terminal, characterized in that, The data acquisition device is used to execute the data acquisition method of the avatar intelligent terminal as described in any one of claims 1 to 14, wherein the avatar intelligent terminal includes a processing module for environmental perception and task planning and a control module for real-time motion control; the processing module is provided with a first circular buffer for caching environmental perception data of each frame; The control module is provided with a second circular buffer for caching control data for each frame; the device includes: The detection unit is used to perform anomaly detection on the embodied intelligent terminal based on target data under multiple detection dimensions using the control module. These multiple detection dimensions include a physical bus dimension, a processing module output dimension, and an interaction dimension, with different detection dimensions corresponding to different anomaly scenarios. The target data under the physical bus dimension includes motor current and absolute acceleration output by the inertial measurement unit over multiple consecutive cycles. The target data under the processing module output dimension includes the target's movement trajectory and positioning increment within a preset time window. The target data under the interaction dimension includes whether an interactive control command has been received. The generation unit is used to generate a data acquisition instruction based on the detection time if any abnormality is detected in any target detection dimension, and send it to the processing module. The first acquisition unit is used to acquire first cached data located within the first time window corresponding to the detection time from the second circular buffer using the control module; the first time window is obtained by backtracking a first duration and appending a second duration, respectively, starting from the data frame time corresponding to the detection time. The second acquisition unit is used to utilize the processing module in response to the data acquisition command, determine a second time window based on the detection time, and acquire second cached data located within the second time window from the first annular buffer. The transmission unit is used to transmit the first cached data, the second cached data, and the scene labels corresponding to the target detection dimension to cloud storage.

Images