Robot data collection system and method based on intelligent terminal

Abstract

The application provides a kind of robot data acquisition system and method based on intelligent terminal, and the system includes: through the application program running on intelligent mobile terminal, the data stream containing position posture and sensor data is collected;And utilize a real-time anomaly detection system to the data stream is analyzed in real time, to identify the preset abnormal event.The real-time anomaly detection system includes: visual feature stability detection unit, for detecting the stability of instant positioning and map construction;Space operation range constraint unit, for detecting whether intelligent mobile terminal exceeds the preset operation range;And kinematics consistency check unit, for distinguishing the real movement of intelligent mobile terminal and the position jump caused by positioning algorithm error.This application can significantly improve data efficiency, reduce post-processing cost, and ensure the quality and reliability of trajectory data by evaluating and marking data quality in real time at the acquisition end and providing real-time feedback to operators.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and in particular to a robot data acquisition system and method based on a smart mobile terminal. Background Technology

[0002] With the development of embodied intelligence technology, training robots by imitating human demonstrations, i.e., behavioral cloning, has become an important approach. Traditional robot behavioral cloning data collection mainly relies on remote operation on real robots. This method is costly, has strict site requirements, and is complex to deploy, limiting the scale and diversity of data collection.

[0003] To lower the barrier to data acquisition, several portable or handheld data acquisition solutions have emerged in the current technology. For example, operators can use handheld devices integrated with sensors such as cameras to record their demonstration actions, thereby converting the operator's hand movements and grasping intentions into data that can be used to train robots. These solutions reduce acquisition costs to some extent and improve the flexibility of data acquisition.

[0004] However, the aforementioned handheld data acquisition solutions still have significant drawbacks in practical applications. The data acquisition process is typically "offline," meaning the operator cannot assess the data quality in real time. For example, the operator cannot determine whether the current demonstration action exceeds the target robot's reachable working range, cannot perceive whether the visual features of the acquisition environment are rich enough to support stable real-time localization and map building, and cannot detect localization jumps or data frame drops caused by algorithm errors or system load. These problems result in the acquired data often containing a large number of invalid or abnormal segments, leading to low data efficiency and incurring significant costs and time overhead for subsequent data filtering and cleaning. Summary of the Invention

[0005] In view of the deficiencies in the prior art, the purpose of this invention is to provide a robot data acquisition system and method based on a smart terminal.

[0006] According to the present invention, a robot data acquisition system based on a smart mobile terminal is provided, the system comprising: The intelligent mobile terminal runs an application for collecting a data stream containing position, attitude, and sensor data; and A real-time anomaly detection system is configured to perform real-time analysis on the data stream to identify anomaly events of a preset type; The real-time anomaly detection system includes: A visual feature stability detection unit is used to detect the stability of instant localization and map construction based on the environmental visual features in the data stream. A spatial operation range constraint unit is used to detect whether the intelligent mobile terminal exceeds a preset operation range based on its position and orientation in the data stream; and The kinematic consistency verification unit is used to distinguish between the actual movement of the smart mobile terminal and the position jump caused by the positioning algorithm error, based on the position and attitude in the data stream.

[0007] Preferably, the real-time anomaly detection system further includes: The time-domain continuity monitoring unit is used to detect data frame loss events based on the timestamp of the data stream.

[0008] Preferably, the visual feature stability detection unit is configured as follows: The number of environmental feature points is counted within a time sliding window. When the percentage of frames with the number of feature points below a preset threshold exceeds a preset proportion, a stability anomaly event is determined to have occurred.

[0009] Preferably, the spatial operation range constraint unit is configured as follows: Calculate the Euclidean norm of the current position of the smart mobile terminal relative to the origin of the coordinate system. When the Euclidean norm is greater than a preset distance threshold, an abnormal event exceeding the operating range is determined to have occurred.

[0010] Preferably, the kinematic consistency verification unit is configured as follows: By comparing the estimated velocity obtained by filtering and smoothing the position and attitude with the instantaneous velocity calculated based on the position and attitude, it is determined whether there is an abnormal position jump event.

[0011] Preferably, the kinematic consistency verification unit is further configured as follows: The comparison described in claim 5 is performed separately in forward and reverse chronological order to obtain forward and reverse difference values; and Based on the average of the positive difference and the negative difference, it is determined whether the position jump anomaly event exists.

[0012] Preferably, the filtering and smoothing process is implemented using a Kalman filter.

[0013] Preferably, the system is further configured to: When the abnormal event is identified, a corresponding abnormal label is generated, and the abnormal label is associated with the data in the data stream that corresponds to the abnormal event in time and stored together.

[0014] According to the present invention, a robot data acquisition method based on a smart mobile terminal is provided, the method comprising: A data stream containing position, attitude, and sensor data is collected by running an application on the smart mobile terminal. The data stream is analyzed in real time to identify abnormal events of a preset type; The real-time analysis steps include: Perform visual feature stability detection, based on the environmental visual features in the data stream, to detect the stability of instantaneous localization and map construction; The system executes spatial operation range constraints, and based on the position and orientation in the data stream, detects whether the intelligent mobile terminal exceeds the preset operation range; and Perform kinematic consistency verification to distinguish between the actual movement of the smart mobile terminal and position jumps caused by positioning algorithm errors, based on the position and attitude in the data stream.

[0015] Preferably, the step of performing kinematic consistency verification includes: The position and attitude are filtered and smoothed in ascending time sequence to obtain a positive estimated velocity, and the difference between the positive estimated velocity and the instantaneous velocity calculated based on the position and attitude is calculated to obtain a positive difference index. The position and attitude are filtered and smoothed in reverse chronological order to obtain the inverse estimated velocity. The difference between the inverse estimated velocity and the instantaneous velocity calculated based on the position and attitude is then calculated to obtain the inverse difference index. Based on the average of the positive difference index and the negative difference index, it is determined whether there is an abnormal location jump event.

[0016] Compared with the prior art, the present invention has the following beneficial effects: 1. This application enables real-time, multi-dimensional evaluation and labeling of data quality at the data acquisition end, which can filter or label a large amount of invalid data caused by various reasons such as insufficient environmental texture, exceeding the working range, system lag and frame loss, and positioning algorithm errors, thereby significantly reducing the manual and time costs required for subsequent data screening and cleaning.

[0017] 2. This application provides operators with real-time status feedback and abnormal warnings through a user interface. Operators can immediately adjust their operations or data collection environment, realizing a closed-loop operation of "collecting and inspecting simultaneously", avoiding a lot of ineffective work and improving the interactivity and efficiency of data collection.

[0018] 3. The kinematic consistency verification scheme proposed in this application can effectively distinguish between real rapid movement and "position jumps" caused by positioning algorithm errors, accurately identify and mark non-physical positioning errors that are extremely harmful to robot learning, and ensure the quality and reliability of the trajectory data used for final training. Attached Figure Description

[0019] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This application provides a schematic diagram of the architecture of a robot data acquisition system. Figure 2 A flowchart illustrating a robot data acquisition method provided in an embodiment of this application; Figure 3 A schematic diagram of the kinematic consistency verification unit provided in the embodiments of this application; Figure 4 A schematic diagram of the user interface provided in an embodiment of this application.

[0020] Explanation of reference numerals in the attached figures: Detailed Implementation

[0021] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0022] Example 1 This embodiment provides a robot data acquisition system and method based on a smart mobile terminal. In one embodiment of this application, by introducing a real-time, multi-dimensional quality monitoring mechanism during the data acquisition process, the aim is to improve the effectiveness of the data required for robot behavior cloning from the source and reduce the cost of subsequent data processing.

[0023] Figure 1 This is a schematic diagram of the overall architecture of a robot data acquisition system provided in an embodiment of this application. The system can be built on a smart mobile terminal, such as a smartphone integrating a camera, LiDAR sensor, and inertial measurement unit. The dedicated application installed and running on this smartphone constitutes the core of the system.

[0024] During system operation, a multimodal data stream is first acquired via data acquisition source 10. This data acquisition source 10 may include various sensors built into the smart mobile terminal, such as cameras for capturing environmental visual information, lidar sensors for acquiring depth information, and inertial measurement units for measuring the device's angular velocity and acceleration. In some application scenarios, the data acquisition source 10 may also include external devices connected wirelessly via Bluetooth, such as encoders for synchronously recording the operator's grasping intentions or grippers with sensors. The acquired data stream is a time-series data stream, containing color images for each frame, depth maps, the device's six-degree-of-freedom position and attitude information (calculated in real-time by a simultaneous localization and mapping algorithm), and possible external device status data.

[0025] The acquired data stream is then sent to the real-time anomaly detection core 20. It is understood that this real-time anomaly detection core 20 is responsible for real-time analysis of the input data stream to identify various preset types of abnormal events. To achieve efficient parallel processing, the real-time anomaly detection core 20 integrates multiple dedicated detection units. In this embodiment, as... Figure 1 As shown, these units include: a visual feature stability detection unit 31, a spatial operation range constraint unit 32, a temporal continuity monitoring unit 33, and a kinematic consistency verification unit 34. These four units work in parallel, monitoring data quality from different dimensions.

[0026] Once any detection unit identifies an abnormal event, the system calls the abnormal label generation module 40. Based on the output of the detection unit, the module 40 generates a structured abnormal label corresponding to the abnormal type, such as "insufficient feature points", "out of workspace", "long-term frame drop" or "SLAM jump".

[0027] Correspondingly, the system also has real-time feedback capabilities. The current status of each detection unit (whether normal or abnormal) is sent to the user interface 50. This user interface 50 can specifically be an application display interface on the screen of a smart mobile terminal, which can present the status information to the operator in an intuitive way (such as color-changing icons, text prompts, etc.), thereby realizing a closed-loop operation of "sampling and inspection simultaneously".

[0028] Finally, the raw data stream, along with the anomaly labels generated by the anomaly label generation module 40, is sent to the data collection and storage module 60. This module 60 is responsible for accurately aligning and associating the anomaly labels with the data frames at the time of the anomaly using timestamps, and then writing the complete dataset containing the raw data and quality labels into the local storage of the smart mobile terminal for subsequent analysis and training.

[0029] The following will combine Figure 2This embodiment further elaborates on the specific process of the robot data acquisition method. This method closely corresponds to the system described above. The method begins with step S201, acquiring a multimodal data stream, which corresponds to the process of the system continuously receiving sensor data from the data acquisition source 10. In this embodiment, the target frame rate for data acquisition can be set to 30 frames per second.

[0030] Subsequently, in step S202, real-time anomaly detection is performed in parallel. Specifically, this step includes performing visual feature stability detection step S202a, performing spatial operation range constraint detection step S202b, performing temporal continuity monitoring step S202c, and performing kinematic consistency verification step S202d. These four sub-steps are respectively performed by… Figure 1 The visual feature stability detection unit 31, spatial operation range constraint unit 32, temporal continuity monitoring unit 33, and kinematic consistency verification unit 34 are used to perform the operation.

[0031] Within the processing cycle of each data frame, step S203 determines whether an abnormal event exists. This step summarizes the judgment results of all detection units. If any detection unit reports an abnormality (i.e., along the "yes" path), the process proceeds to step S204, where an abnormality tag is generated and associated. In this step, the abnormality tag generation module 40 creates the corresponding tag and prepares to associate it with the current data frame. If none of the detection units report an abnormality (i.e., along the "no" path), step S204 is skipped.

[0032] Regardless of whether any anomalies exist, the process continues to step S205, providing real-time status feedback to the user interface. In this step, the user interface 50 will update its display to inform the operator of the current status of each data quality indicator.

[0033] Finally, the process proceeds to step S206, where the data frame and associated anomaly tags are stored. The data collection and storage module 60 packages the sensor data of the current frame, along with any anomaly tags that may have been generated in step S204, and writes them into storage. Afterward, the system returns to step S201 to process the next frame of data in a loop until the operator stops the data acquisition task.

[0034] Specifically, the implementation details of each detection unit are described below: Visual Feature Stability Detection Unit 31: This unit aims to provide early warning of potential positioning failures or drifts in the real-time localization and mapping (RTL) system due to insufficient environmental texture features. Internally, unit 31 maintains a time-sliding window; in this embodiment, its length T can be set to 3 seconds. For each frame of data collected by the application, unit 31 obtains the number of successfully tracked environmental visual feature points in the current frame from the RTL subsystem, denoted as... At the same time, a preset threshold for the number of feature points is set. (In this embodiment, the number is set to 20 based on experience) and a frame percentage threshold α (set to 90%). In each processing cycle, unit 31 checks all data frames within the most recent 3-second sliding window that meet the condition " The proportion of frames with a value less than 20 is considered. If this proportion exceeds 90%, the unit determines that the visual texture features of the current environment are severely insufficient, posing a very high risk of localization loss, and outputs an abnormal signal of "insufficient feature points".

[0035] Spatial Operation Range Constraint Unit 32: This unit ensures that the operator's demonstration actions always remain within the physical workspace that the target robot can actually reproduce. At the start of the data acquisition task, the system defines the initial position of the intelligent mobile terminal as the origin O(0,0,0) of the world coordinate system. During the acquisition process, this unit 32 acquires the device's current position coordinates output by the instant localization and mapping system in real time. ( , , ), and calculate the Euclidean norm of the device's current position relative to the origin, i.e., the straight-line distance. The calculation formula is as follows:

[0036] In this embodiment, a distance threshold is preset. Its value is usually set based on the target robot's working radius; here, it's set to 1.0 meter. When the calculated distance... When the distance exceeds 1.0 meter, the unit 32 determines that the equipment has exceeded the preset operating radius and immediately outputs an abnormal signal of "out of working space".

[0037] Temporal continuity monitoring unit 33: This unit detects frame loss during data acquisition caused by factors such as processor overload, system scheduling delay, or background task interference in the smart mobile terminal. Since the target acquisition frame rate in this embodiment is f=30 frames per second, theoretically, the time interval between two adjacent frames should be... =1 / 30 ≈ 33.3 milliseconds. This unit 33 compares the timestamp of the current frame. With the previous frame timestamp The difference Δt = - To identify anomalies.

[0038] The specific judgment logic is divided into two types: First, severe frame drop judgment: If a frame interval Δt is detected to exceed 2.5 times the theoretical interval, i.e., Δt > 2.5 × 33.3ms ≈ 83.3ms, it indicates that a significant and prolonged stutter has occurred, and the system judges it as a "long-term frame drop" anomaly. Second, frequent jitter judgment: If within a preset short time window (e.g., 1 second), two or more consecutive occurrences of Δt > 1.5 × 33.3ms occur, the system judges it as a "long-term frame drop" anomaly. Even if a single stutter is not severe (approximately 50ms), frequent jitter can still disrupt the timing smoothness of the data, and the system will determine it as a "continuous frame drop" anomaly. Once any of the above situations is detected, unit 33 will output the corresponding abnormal signal.

[0039] Kinematic Consistency Verification Unit 34: As a core design element of this embodiment, this unit aims to accurately distinguish between two phenomena that appear similar but are fundamentally different: the operator's genuine rapid movement and the "position jump" caused by errors in the instantaneous localization and mapping algorithm. The latter is a non-physical, instantaneous coordinate change that can severely interfere with subsequent robot learning. For the principles of this unit, please refer to [link to relevant documentation]. Figure 3 .like Figure 3 As shown, the kinematic consistency verification unit 34 receives the raw position data input 341, which is the raw position coordinate sequence output by the real-time positioning and mapping system, which may contain noise and jumps. For verification, the data is sent to the bidirectional verification logic module 348 and processed along two parallel paths. The first path is the state estimation path. The raw position data input 341 is fed into a Kalman filter 342. In this embodiment, a linear Kalman filter can be used. This filter recursively predicts and updates the position data based on a kinematic model (e.g., a uniform velocity or uniform acceleration model), thereby filtering out high-frequency noise and outputting a smooth, physically consistent estimated velocity 343, denoted as . The second path is the instantaneous observation path. The raw position data input 341 is fed into a differential calculation module 344. This module calculates the instantaneous velocity 345 using simple backward differential, denoted as... The calculation formula is as follows:

[0040] in and These are the original observation positions at the current time and the previous sampling time, respectively, and δt is the time difference between the two sampling times. This instantaneous velocity... All details of the original data are preserved, including real rapid motion and potential positional jumps. Subsequently, the estimated velocities 343 and instantaneous velocities 345 from the two paths are simultaneously fed into the difference measurement and comparison module 346. This module first calculates a difference metric A to quantify the deviation between the instantaneous velocity and the smoothed estimated velocity. The physical meaning of this metric can be approximated as "pseudo-acceleration" caused by non-kinematic factors (such as algorithmic errors), and its calculation formula is:

[0041] Where f is the frame rate. When a position change occurs... It will become extremely large in an instant, and Due to the smoothing effect of the filter, the change is relatively delayed and gradual, leading to a sharp increase in the A value. To further improve the robustness of the detection and avoid misjudging real violent movements as jumps, this embodiment introduces a bidirectional verification logic module 348. Specifically, the difference measurement and comparison module 346 performs two calculations: First, it inputs a segment of position data into the Kalman filter 342 and the difference calculation module 344 in ascending time sequence (from past to present) to obtain a positive difference index. The second time, the same location data is input in reverse chronological order (from current to past) to obtain a reverse difference index. The final determination is based on comparing the average of these two indicators with a preset threshold. (In this embodiment, Set to 15.0). When the condition is met ( + ) / 2> Only then does the system finally determine that a jump in real-time localization and map building location has occurred, and the SLAM jump flag output module 347 outputs an anomaly signal. This two-way verification method can more reliably smooth the data, allowing errors in both directions to partially cancel each other out during real motion. At real jump points, significant abrupt changes are detected in both directions, thereby improving the distinguishability.

[0042] Finally, for details on the implementation of user interface 50, please refer to [link / reference]. Figure 4 . Figure 4This is a schematic diagram of the application user interface in this embodiment. During data acquisition, the main area of ​​the screen can display the real-time camera view 410 to facilitate the operator's observation of the object being operated on and the environment. A semi-transparent status indicator area 420 is overlaid on this view. This area contains four independent indicators, corresponding to the status of four detection units: visual feature status indicator 421, spatial range status indicator 422, temporal continuity status indicator 423, and kinematic consistency status indicator 424. Under normal circumstances, these indicators are displayed in green. Once any detection unit, such as the visual feature stability detection unit 31, detects an anomaly (e.g., insufficient feature points), the corresponding visual feature status indicator 421 immediately turns red, possibly accompanied by the text prompt "Insufficient environmental texture." This immediate visual feedback guides the operator to take immediate corrective measures, such as moving the device to an area with richer texture, thereby avoiding continued acquisition under invalid conditions and greatly improving the efficiency of data acquisition and the usability of the final data.

[0043] Example 2 As an optional implementation of Embodiment 1, this embodiment mainly optimizes the kinematic consistency verification unit 34 to adapt to more complex motion scenarios.

[0044] It should be noted that in Embodiment 1, the kinematic consistency verification unit 34 internally employs a linear Kalman filter. Linear Kalman filters perform well when an object undergoes approximately linear motion (such as uniform or uniformly accelerated motion). However, when an operator performs highly nonlinear and complex actions using a handheld smart mobile terminal, such as a spiral trajectory involving rapid rotation and variable acceleration, the assumptions of the linear model may no longer hold, leading to a decrease in the filter's tracking accuracy. This may affect the estimated velocity. The deviation from the actual motion trajectory is large, which reduces the sensitivity of detecting positional jumps, and may even misjudge violent real motion as jumps.

[0045] To address this issue, this embodiment replaces the linear Kalman filter in the kinematic consistency verification unit 34 with an extended Kalman filter. The overall system architecture is as follows: Figure 1 The core modifications remain unchanged, with the Kalman filter 342 module inside the kinematic consistency verification unit 34 remaining the same.

[0046] Specifically, when using the Extended Kalman Filter (EKF), the system's state transition and observation models are no longer simple linear matrices, but rather nonlinear functions. For example, the state transition function f(x) and observation function h(x) can describe more complex motion dynamics. In the filter's prediction and update steps, the system needs to be locally linearized by calculating the Jacobian matrix (i.e., the first-order partial derivative matrix) of these nonlinear functions at the current state estimation point. Furthermore, the state vector X can be further extended beyond position and velocity to include higher-order kinematic variables such as acceleration, in order to construct a higher-order, more accurate motion model.

[0047] Its working process can be described as follows: When the operator performs a complex demonstration maneuver involving rapid steering and acceleration / deceleration, the extended Kalman filter in this embodiment is able to fit this nonlinear trajectory better than the linear Kalman filter in Embodiment 1. Therefore, its output estimated velocity... This will reflect the true movement trend more smoothly and accurately. When the real-time localization and mapping algorithm experiences positional jumps due to scene confusion or calculation errors during this process, the instantaneous velocity calculated from the original observations will be affected. This will be compared with a more accurate speed estimate This results in more significant and sharp differences between them. This allows the difference measurement index A calculated by the difference measurement and comparison module 346 to exceed the preset jump threshold more sensitively and reliably. .

[0048] Compared to Example 1, this example employs an extended Kalman filter, which improves the adaptability of the kinematic model to complex nonlinear motion. Its beneficial effect is that, when dealing with highly dynamic and nonlinear user operations, it can more accurately estimate the actual motion state of the device. This results in a lower false alarm rate (i.e., reducing the probability of misjudging real, drastic motion as a jump) and a lower false negative rate (i.e., improving the ability to detect jumps occurring during complex motion processes) when detecting instantaneous localization and map-building position jumps, further enhancing the robustness and reliability of trajectory data quality monitoring.

[0049] Example 3 This embodiment is another variation of embodiment 1. Its core lies in the introduction of an adaptive threshold mechanism to replace the fixed threshold used in embodiment 1, thereby enabling the data acquisition system to better adapt to different application scenarios and hardware targets.

[0050] It is understandable that in Embodiment 1, the judgment thresholds of each detection unit, such as the distance threshold of the spatial operation range constraint unit 32, are... (1.0 meter) and the threshold for the number of feature points of the visual feature stability detection unit 31 (20 values), all preset fixed values. However, in practical applications, different robot models have drastically different workspace ranges, and different data acquisition tasks have different requirements for positioning accuracy. Using a "one-size-fits-all" fixed threshold may be too lenient in some scenarios (collecting data that cannot be reproduced), while being too strict in other scenarios (restricting necessary operations).

[0051] To address this issue, this embodiment improves the software flow and internal logic of the application. Specifically, a task configuration interface is added to the application's user interface 50 before the data acquisition task begins. On this interface, the operator can select the target robot model for this data acquisition, for example, from a list containing common collaborative robots such as "UR5", "FrankaEmika Panda", and "KUKA LBR iiwa". Additionally, the operator can select the task type, such as "delicate desktop operation" or "large-area navigation object retrieval".

[0052] Accordingly, a parameter database is also built into the application. This database stores parameter configuration sets associated with different robot models and task types. For example, for the "Franka Emika Panda" robot, the database stores its officially stated maximum working radius of approximately 0.85 meters; for the "UR5" robot, its working radius is 0.85 meters. For "desktop fine manipulation" tasks, due to the need for high-precision positioning and reconstruction, the corresponding feature point number threshold... It can be set to a higher value, such as 50; however, for the "large-scale navigation and object retrieval" task, the requirement for local positioning accuracy is relatively low, and this threshold can be appropriately relaxed to 20.

[0053] The system's workflow can be described as follows: Before the operator starts the data acquisition task, they first select the target robot model "Franka Emika Panda" and the task type "Desktop Fine Operation" in the configuration interface. After receiving the user's selection, the application queries and loads the corresponding parameters from the internal parameter database. At this time, the distance threshold of the spatial operation range constraint unit 32... It will be automatically set to 0.85 meters, and the threshold for the number of feature points of the visual feature stability detection unit 31. The value is then set to 50. During the subsequent data acquisition process, the spatial operation range constraint unit 32 will calculate the distance between the device position and the origin in real time and compare it with 0.85 meters. Once the operator's arm extends beyond 0.85 meters, the system will immediately issue an "out of workspace" alarm through the user interface 50. At the same time, the visual feature stability detection unit 31 will use 50 more stringent feature points as the judgment standard to ensure that the real-time positioning and mapping system is always in a high-quality positioning state throughout the entire detailed operation demonstration.

[0054] By introducing this adaptive threshold strategy, this embodiment achieves refined data quality control for specific application scenarios. It ensures that the collected demonstration motion data strictly conforms to the physical constraints of the target robot, avoiding invalid data caused by workspace mismatch. Simultaneously, by adjusting quality requirements according to task type, a balance between resources and accuracy is achieved. This results in a higher degree of matching between the final collected dataset and the target application scenario, eliminating the need for cumbersome post-processing scaling or filtering, thereby further improving the direct usability of the data and training efficiency.

[0055] Example 4 This embodiment is another variation of Embodiment 1, which focuses on improving the monitoring algorithm of the temporal continuity monitoring unit 33, aiming to improve the accuracy and robustness of frame loss event detection.

[0056] In Example 1, the temporal continuity monitoring unit 33 employs a threshold judgment method based on a fixed multiple relationship. For example, when the frame interval exceeds 1.5 or 2.5 times the theoretical value, it is judged as abnormal. This method is simple and intuitive, but may be too sensitive in some cases. The operating system of modern smart mobile terminals is a complex multi-tasking environment, and normal system scheduling, background application activities, etc., may cause small, occasional jitter in the data acquisition frame rate. Using a fixed threshold may falsely report these normal jitters that do not affect the overall data quality as abnormal frame drops, thereby generating unnecessary warnings, interfering with the operator, and incorrectly marking data.

[0057] To more accurately distinguish between normal jitter and genuine abnormal frame drops, this embodiment employs a statistical anomaly detection method to reconstruct the internal logic of the temporal continuity monitoring unit 33. The specific implementation is as follows: The temporal continuity monitoring unit 33 internally maintains a sliding window or queue for storing the frame interval time Δt of the most recent N frames (e.g., N=100). This window records historical information about the recent system frame rate performance. For each new frame acquired, the system first calculates its interval time with the previous frame. Then, the unit calculates and updates the moving average of all 100 frame intervals within the sliding window in real time. and standard deviation Among them, the moving average It reflects the recent average frame interval, while the standard deviation This quantifies the fluctuation or dispersion of recent frame intervals. The logic for judging anomalies is no longer based on comparison with fixed theoretical values, but rather on statistical distribution. If the newly calculated current frame interval... The following conditions must be met: > +k* The frame is then determined to be an abnormal frame loss. Here, k is an adjustable parameter representing a multiple of the standard deviation, typically set to 3. Statistically, this is known as the "3-sigma" criterion, which states that a data point deviating from the mean by more than 3 standard deviations can be considered a low-probability event, i.e., an outlier.

[0058] The following example illustrates the working process: Assume the system is running under normal, low load conditions, with a stable frame rate of approximately 30 frames per second. In this case, the frame interval Δt may fluctuate slightly around 33.3ms, for example, between 30ms and 36ms. Under these circumstances, the moving average calculated by the sliding window... It will stabilize at approximately 33ms, with a standard deviation of... The value will be very small, for example, 1ms. According to the 3-sigma criterion, the anomaly threshold is approximately 33 + 3 * 1 = 36ms. This means that as long as the frame interval jitter is within this range, the system will consider it normal. However, if a resource-intensive task (such as a software update or large data synchronization) suddenly starts in the background of the smartphone, causing the processing of this application to be preempted, resulting in a noticeable lag, the processing time of a certain frame may spike to 100ms. Therefore, the newly calculated... The threshold will be 100ms. Since 100ms is much larger than the currently calculated abnormal threshold of 36ms, the system will immediately determine that this is an abnormal frame drop event that significantly deviates from recent normal performance and generate an abnormal label such as "long-term frame drop".

[0059] Compared to Example 1, the statistical method used in this example can dynamically adapt to the current operating state of the system. It can effectively ignore normal frame rate jitter within an acceptable range, focusing only on "outlier" events that statistically significantly deviate from recent performance. This greatly improves the accuracy of frame drop detection, reduces false alarms caused by normal system jitter, makes temporal continuity analysis more robust and reliable, and ensures that only stutters that truly affect data quality are flagged.

[0060] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A robot data acquisition system based on an intelligent mobile terminal, characterized in that, The system includes: The intelligent mobile terminal runs an application for collecting a data stream containing position, attitude, and sensor data; and A real-time anomaly detection system is configured to perform real-time analysis on the data stream to identify anomaly events of a preset type; The real-time anomaly detection system includes: A visual feature stability detection unit is used to detect the stability of instant localization and map construction based on the environmental visual features in the data stream. A spatial operation range constraint unit is used to detect whether the intelligent mobile terminal exceeds a preset operation range based on its position and orientation in the data stream; and A kinematic consistency verification unit is used to distinguish between the actual movement of the smart mobile terminal and the position jump caused by the positioning algorithm error, based on the position and attitude in the data stream. The kinematic consistency verification unit is configured as follows: By comparing the estimated velocity obtained by filtering and smoothing the position and attitude with the instantaneous velocity calculated based on the position and attitude, it is determined whether there is an abnormal position change event. The comparisons were performed separately in ascending and descending chronological order to obtain forward and reverse difference values; and Based on the average of the positive difference and the negative difference, it is determined whether the position jump anomaly event exists.

2. The robot data acquisition system based on a smart mobile terminal according to claim 1, characterized in that, The real-time anomaly detection system also includes: The time-domain continuity monitoring unit is used to detect data frame loss events based on the timestamp of the data stream.

3. The robot data acquisition system based on an intelligent mobile terminal according to claim 1, characterized in that, The visual feature stability detection unit is configured as follows: The number of environmental feature points is counted within a time sliding window. When the percentage of frames with the number of feature points below a preset threshold exceeds a preset proportion, a stability anomaly event is determined to have occurred.

4. The robot data acquisition system based on an intelligent mobile terminal according to claim 1, characterized in that, The spatial operation range constraint unit is configured as follows: Calculate the Euclidean norm of the current position of the smart mobile terminal relative to the origin of the coordinate system. When the Euclidean norm is greater than a preset distance threshold, an abnormal event exceeding the operating range is determined to have occurred.

5. The robot data acquisition system based on an intelligent mobile terminal according to claim 1, characterized in that, The filtering and smoothing process is implemented using a Kalman filter.

6. The robot data acquisition system based on an intelligent mobile terminal according to claim 1, characterized in that, The system is also configured to: When the abnormal event is identified, a corresponding abnormal label is generated, and the abnormal label is associated with the data in the data stream that corresponds to the abnormal event in time and stored together.

7. A robot data acquisition method based on a smart mobile terminal, characterized in that, The method includes: A data stream containing position, attitude, and sensor data is collected by running an application on the smart mobile terminal. The data stream is analyzed in real time to identify abnormal events of a preset type; The real-time analysis steps include: Perform visual feature stability detection, based on the environmental visual features in the data stream, to detect the stability of instantaneous localization and map construction; The system executes spatial operation range constraints, and based on the position and orientation in the data stream, detects whether the intelligent mobile terminal exceeds the preset operation range; and Perform kinematic consistency verification to distinguish between the actual movement of the smart mobile terminal and position jumps caused by positioning algorithm errors, based on the position and attitude in the data stream. The steps for performing the kinematic consistency check include: The position and attitude are filtered and smoothed in ascending time sequence to obtain a positive estimated velocity, and the difference between the positive estimated velocity and the instantaneous velocity calculated based on the position and attitude is calculated to obtain a positive difference index. The position and attitude are filtered and smoothed in reverse chronological order to obtain the inverse estimated velocity. The difference between the inverse estimated velocity and the instantaneous velocity calculated based on the position and attitude is then calculated to obtain the inverse difference index. Based on the average of the positive difference index and the negative difference index, it is determined whether there is an abnormal location jump event.

Images