Robot correction method and device, robot and computer readable storage medium

Abstract

The invention discloses a robot correction method and device, a robot and a computer readable storage medium. The method comprises the steps that multi-dimensional original sensing data are collected through a plurality of heterogeneous sensors; obtaining synchronous and aligned multi-source sensing information based on the multi-dimensional original sensing data; constructing a multiple directed graph based on the multi-source sensing information; based on the multiple directed graphs, generating a node sequence by adopting a random walk sampling strategy with bias; the random walk sampling strategy dynamically adjusts the sampling probability of neighborhood nodes according to the data quality and timestamp freshness of the multi-source sensing information associated with each edge; training a Skip-gram model according to the node sequence to obtain a low-dimensional embedded vector of each node; and correcting a motion track or an operation strategy of the robot based on the low-dimensional embedded vector. Therefore, the reliability and the adaptability of track or strategy correction are improved by enabling node embedding to be more likely to reflect high-quality and recent operation contexts.

Description

Technical Field

[0001] This application relates to the technical field of robot control, and more specifically, to a robot calibration method, apparatus, robot, and computer-readable storage medium. Background Technology

[0002] When industrial robots perform precision assembly, insertion, or grasping tasks, operational accuracy highly depends on the accurate perception and modeling of their own state and interactions with the environment. Current methods typically build operational process models based on sensor data and then perform behavioral analysis or control optimization accordingly.

[0003] However, in practical applications, the lack of preliminary fusion of sensor data makes it difficult to obtain consistent and reliable state representations. Furthermore, existing modeling methods typically treat state transitions as indifferent logical connections during operation, failing to effectively reflect the performance differences of the same transition in different execution instances (e.g., successful completion versus deviation attempts).

[0004] This makes the behavioral patterns learned from historical data lack the ability to distinguish the effectiveness of operations, thus limiting their practicality and robustness in real-time trajectory adjustment or strategy optimization. Summary of the Invention

[0005] In view of the above problems, this application proposes a robot correction method, device, robot and computer-readable storage medium, which can perform preliminary fusion of sensor data and automatically identify and structure the high-quality and recent effective behavior patterns in the robot's historical operation data, thereby driving adaptive and semantically aware trajectory or strategy correction for the current task, significantly improving the success rate and robustness of precision operation tasks.

[0006] In a first aspect, embodiments of this application provide a robot correction method. The robot body and its environment are equipped with multiple heterogeneous sensors. The method includes: collecting multidimensional raw sensor data through multiple heterogeneous sensors during robot task execution; performing spatiotemporal calibration and noise filtering on the multidimensional raw sensor data to obtain synchronized and aligned multi-source sensor information; constructing a multi-directed graph based on the multi-source sensor information, where nodes represent robot operation states, edges represent state transition instances, and each edge is accompanied by its corresponding multi-source sensor information; generating a node sequence for graph embedding training based on the multi-directed graph using a biased random walk sampling strategy; dynamically adjusting the sampling probability of neighboring nodes according to the data quality and timestamp freshness of the multi-source sensor information associated with each edge; training a Skip-gram model based on the node sequence to obtain a low-dimensional embedding vector for each node; and correcting the robot's motion trajectory or operation strategy based on the low-dimensional embedding vector.

[0007] Secondly, embodiments of this application also provide a robot correction device. The robot body and its environment are equipped with multiple heterogeneous sensors. The device includes: a data acquisition module for acquiring multidimensional raw sensor data through multiple heterogeneous sensors during robot task execution; a preprocessing module for performing spatiotemporal calibration and noise filtering on the multidimensional raw sensor data to obtain synchronized and aligned multi-source sensor information; a construction module for constructing a multi-directed graph based on the multi-source sensor information, where nodes represent robot operation states, edges represent state transition instances, and each edge is accompanied by its corresponding multi-source sensor information; a generation module for generating a node sequence for graph embedding training based on the multi-directed graph using a biased random walk sampling strategy; the random walk sampling strategy dynamically adjusts the sampling probability of neighboring nodes according to the data quality and timestamp freshness of the multi-source sensor information associated with each edge; a training module for training a Skip-gram model based on the node sequence to obtain a low-dimensional embedding vector for each node; and an execution module for correcting the robot's motion trajectory or operation strategy based on the low-dimensional embedding vector.

[0008] Thirdly, embodiments of this application also provide a robot, including a processor, a memory, and one or more applications; the one or more applications are stored in the memory and configured to be executed by the processor to implement the above-described robot correction method.

[0009] Fourthly, embodiments of this application also provide a computer-readable storage medium storing program code, wherein the above-described robot correction method is executed when the program code is run by a processor.

[0010] The technical solution provided in this application includes the following method: during the robot's task execution, multidimensional raw sensing data is collected through multiple heterogeneous sensors; spatiotemporal calibration and noise filtering are performed on the multidimensional raw sensing data to obtain synchronized and aligned multi-source sensing information; based on the multi-source sensing information, a multi-directed graph is constructed, where nodes represent robot operation states, edges represent state transition instances, and each edge is accompanied by its corresponding multi-source sensing information; based on the multi-directed graph, a biased random walk sampling strategy is adopted to generate a node sequence for graph embedding training; the random walk sampling strategy dynamically adjusts the sampling probability of neighboring nodes according to the data quality and timestamp freshness of the multi-source sensing information associated with each edge; the Skip-gram model is trained according to the node sequence to obtain a low-dimensional embedding vector for each node; and the robot's motion trajectory or operation strategy is corrected based on the low-dimensional embedding vector. Therefore, by fusing multi-source sensor information to construct a graph model and performing bias sampling based on the quality and timeliness of the sensor data associated with each state transition, the node embedding is more inclined to reflect high-quality, recent operational context, thereby improving the reliability and adaptability of trajectory or strategy correction. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments and drawings obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0012] Figure 1 A flowchart illustrating a robot calibration method provided in an embodiment of this application is shown.

[0013] Figure 2 A schematic diagram of the structure of a robot correction device provided in an embodiment of this application is shown.

[0014] Figure 3 A schematic diagram of the structure of a robot provided in an embodiment of this application is shown.

[0015] Figure 4 This illustration shows a schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of this application. Detailed Implementation

[0016] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0017] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0018] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0019] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0020] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0021] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0022] When industrial robots perform precision assembly, insertion, or grasping tasks, operational accuracy highly depends on the accurate perception and modeling of their own state and interaction with the environment. Existing methods typically utilize sensor data to construct operational flow models for behavior analysis or control optimization. However, in practical applications, robots are often equipped with various heterogeneous sensors (such as vision, force sensors, and inertial units), and their raw data suffers from problems such as asynchronous sampling, inconsistent coordinate systems, and noise interference. Without effective spatiotemporal alignment and fusion processing, it is difficult to form a consistent and reliable state representation.

[0023] Furthermore, existing modeling methods generally simplify state transitions to indistinguishable topological connections, failing to distinguish the contextual differences of the same transition in different execution instances (e.g., successful completion versus failed attempts due to deviations). This results in the learned behavioral patterns lacking the ability to discriminate the effectiveness of operations, thus limiting their practicality and robustness in real-time trajectory adjustment or policy optimization.

[0024] To address the aforementioned issues, this application provides a robot calibration method, apparatus, robot, and computer-readable storage medium. The robot body and its environment are equipped with multiple heterogeneous sensors. The method includes: collecting multidimensional raw sensor data through multiple heterogeneous sensors during robot task execution; performing spatiotemporal calibration and noise filtering on the multidimensional raw sensor data to obtain synchronized and aligned multi-source sensor information; constructing a multi-directed graph based on the multi-source sensor information, where nodes represent robot operation states, edges represent state transition instances, and each edge is accompanied by its corresponding multi-source sensor information; generating a node sequence for graph embedding training using a biased random walk sampling strategy based on the multi-directed graph; dynamically adjusting the sampling probability of neighboring nodes according to the data quality and timestamp freshness of the multi-source sensor information associated with each edge; training a Skip-gram model based on the node sequence to obtain a low-dimensional embedding vector for each node; and correcting the robot's motion trajectory or operation strategy based on the low-dimensional embedding vector.

[0025] Therefore, by collecting multidimensional raw sensor data during robot task execution and performing spatiotemporal calibration and noise filtering to obtain synchronized and aligned multi-source sensor information, the problem of inconsistent and unreliable state representation caused by the lack of sensor data fusion in existing technologies is effectively solved. Furthermore, by constructing a multi-directed graph based on this multi-source sensor information, each edge representing a state transition instance is attached with its corresponding multi-source sensor information. Combined with a biased random walk strategy—which dynamically adjusts the sampling probability based on the data quality and timestamp freshness of the multi-source sensor information associated with each edge—high-quality and recent effective operation paths are prioritized when generating node sequences for Skip-gram training. Finally, the resulting low-dimensional embedding vector can distinguish the performance differences of different execution instances and correct the robot's motion trajectory or operation strategy accordingly. This overcomes the deficiency of traditional modeling methods that treat state transitions as indifferent connections, resulting in a lack of effective discriminative ability for behavioral patterns, and significantly improves operational accuracy and robustness.

[0026] This invention provides a robot calibration method. The executing entity of this robot calibration method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a robot, a server, or a terminal. In other words, the robot calibration method can be executed by software or hardware installed on a robot, terminal device, or server device. The software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cluster of cloud servers. The server can be a standalone server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0027] Please see Figure 1 , Figure 1 The diagram illustrates a flowchart of a robot calibration method provided in an embodiment of this application. The robot body and its environment are equipped with multiple heterogeneous sensors. For example... Figure 1 As shown, the method may include steps 110 to 160.

[0028] In step 110, during the robot's task execution, multidimensional raw sensing data is collected through multiple heterogeneous sensors.

[0029] In some implementations, heterogeneous sensors can be a collection of various sensors that differ significantly in physical structure, operating principle, output signal type, and information representation method. Heterogeneous sensors can sense and collect various parameters in the robot's operating environment, including but not limited to multi-dimensional information such as position, attitude, velocity, acceleration, force / torque, temperature, humidity, acoustic characteristics, and optical images.

[0030] In one specific implementation, heterogeneous sensors include, but are not limited to: vision sensors (e.g., RGB cameras, depth cameras), force / torque sensors, laser rangefinders, position encoders, inertial sensors (e.g., accelerometers, gyroscopes), tactile sensors, and other sensors.

[0031] In some implementations, the multidimensional raw sensing data can refer to the unprocessed set of raw signals acquired in real time by the aforementioned heterogeneous sensors during task execution. This includes information about different physical quantities captured by different heterogeneous sensors. For example, a vision sensor outputs a sequence of image frames. Another example is a force / torque sensor outputting a six-dimensional force vector. Since each heterogeneous sensor continuously outputs time-series data at its sampling frequency, and some heterogeneous sensors themselves have spatial structures, it also possesses high temporal-spatial dimensionality.

[0032] For example, during a "screw-grabbing" task, the robot simultaneously records a depth image every 30ms, six-dimensional wrist force data every 1ms, joint angles every 5ms, and IMU readings every 10ms—these constitute multi-dimensional raw sensor data.

[0033] Multidimensional raw sensor data typically suffers from asynchronicity (different sampling rates among sensors), noise interference (such as visual occlusion and force sensor drift), and spatiotemporal misalignment, making it unsuitable for direct use in state modeling. Therefore, spatiotemporal calibration and noise filtering are necessary to obtain synchronized and aligned multi-source sensor information. Specifically: In step 120, spatiotemporal calibration and noise filtering are performed on the multidimensional raw sensing data to obtain synchronized and aligned multi-source sensing information.

[0034] In some implementations, multi-source sensing information can be a fusion sensing data set that is temporally synchronized, spatially aligned, and has a high signal-to-noise ratio, obtained by spatiotemporal calibration and noise filtering of multidimensional raw sensing data collected from multiple heterogeneous sensors.

[0035] Multi-source sensor information is no longer limited to a single modality or a single physical quantity, but rather jointly characterizes the operational context of a robot during task execution from multiple sensory dimensions. Multi-source sensor information is not simply a patchwork of raw data, but a fused representation formed after preprocessing, resulting in clearer semantics and higher reliability.

[0036] By obtaining synchronized and aligned multi-source sensing information, the spatiotemporal inconsistencies between heterogeneous sensors can be eliminated, avoiding misjudgments of state caused by asynchronous sampling or coordinate system differences. Furthermore, noise interference can be suppressed, improving the identifiability of key operational signals (e.g., minute contact forces, target edges).

[0037] For example, in a "pin insertion" task, after spatiotemporal calibration and filtering, the system obtains a frame of synchronization data at t=1.25s, including: the 3D position of the pin aligned to the base coordinate system (from vision), and the smoothed end contact force ( =2.1N, =0.3N), the filtered joint angle vector, and the denoised acceleration reading. These pieces of information together constitute the "multi-source sensor information" at that moment.

[0038] Specifically, the robot aligns data streams from different heterogeneous sensors using timestamps. For example, for high-frequency signals (such as force and joint angles), interpolation (such as linear or spline interpolation) or downsampling methods are used to align them with low-frequency signals (such as image frames) on a unified time grid. Alternatively, for systems without hardware synchronization mechanisms, soft synchronization can be achieved using software timestamps combined with buffer queues. Ultimately, all sensor data is organized into synchronized data frames with fixed time steps (such as every 10 ms).

[0039] The robot also unifies the observation data from various heterogeneous sensors to a common reference coordinate system (usually the robot's base coordinate system or the task coordinate system). For example, using a pre-calibrated extrinsic parameter matrix, the object point cloud captured by the depth camera is transformed from the camera coordinate system to the robot's base coordinate system; the force / torque measured by the six-dimensional force sensor is transformed from the end-effector coordinate system to the world coordinate system. This process relies on the rigid body transformation relationship between the sensors and can be obtained through hand-eye calibration, robot kinematics models, or online estimation methods.

[0040] The robot also incorporates appropriate filtering strategies for different types of heterogeneous sensors to suppress random noise and preserve effective signal characteristics. For example, low-pass filters or Kalman filters are used for force / torque signals. Another example is the use of denoising algorithms (such as non-local mean denoising and Gaussian filtering) for image data.

[0041] After time synchronization, spatial alignment, and noise filtering based on the multidimensional raw sensor data, the multidimensional raw sensor data is transformed into multi-source sensor information with consistent structure, time synchronization, spatial alignment, and improved signal-to-noise ratio. The multi-source sensor information can be regarded as a unified multi-channel perception vector sequence. The data frame at each moment completely represents the robot's current operating context, providing reliable input for subsequent operation state recognition and state transition modeling.

[0042] In step 130, a multi-directed graph is constructed based on multi-source sensor information, where nodes represent robot operation states, edges represent state transition instances, and each edge is accompanied by its corresponding multi-source sensor information.

[0043] The construction of the multi-directed graph is based on preprocessed multi-source sensor information. During the robot's task execution, multi-dimensional raw sensor data is continuously collected through multiple heterogeneous sensors, and the timestamp of each data point is recorded. After spatiotemporal calibration and noise filtering, this data forms structured multi-source sensor information.

[0044] Based on this, the discretized behavioral states of the robot during operation are abstracted as nodes in the graph. These nodes mainly correspond to user nodes (i.e., operation state nodes), which are key semantic points that characterize the robot's own operation state (e.g., "approaching the target object", "contact adjustment", "stable grasping", etc.).

[0045] In addition, multidirected graphs can also contain item nodes to represent task objects or interactive entities in the environment (e.g., screws, holes, workpieces, etc.), thus fully depicting the interactive closed loop of "robot - environment - robot".

[0046] In a multidirected graph, directed edges represent instances of actual state transitions from one state to another. Each valid action (e.g., a grab attempt or an insertion action) corresponds to a separate directed edge.

[0047] In particular, the multidirected graph structure employed in this invention allows for multiple parallel edges between the same pair of starting nodes and the same node. This is because, in real-world tasks, a robot may experience the same pair of nodes multiple times (e.g., from "alignment" to "insertion"), but the sensing context is different each time (e.g., one success, one failure due to excessive force), and these differences must be preserved.

[0048] Crucially, each directed edge does not merely represent a topological connection, but explicitly associates the multi-source sensing information corresponding to the moment it occurs. Specifically, for any slave node... Pointing to node The edges have two core attributes: the sensor data aggregation value and the timestamp aggregation value.

[0049] Among them, the sensor data aggregation value represents the set of all multi-source sensor information collected during the state transition or its quality-weighted aggregation result, and the timestamp aggregation value represents the time freshness measure of the sensor data associated with the transition (e.g., the most recent timestamp, the average time decay weight, etc.).

[0050] Therefore, each edge in a multi-directed graph structure actually carries complete operational context evidence, including multimodal information such as visual, force, and kinematic information and their timeliness, so that the graph structure not only reflects the logical order of state transitions, but also contains fine-grained semantics of physical interactions.

[0051] For example, in performing the "pin insertion" task, the robot transitions twice from state A ("pin aligned with hole") to state B ("pin inserted into hole"): the first attempt causes jamming due to angular deviation, corresponding to the edge... Related Includes high lateral forces ( ) and visual tilt angle (8°). The earlier timestamp; the second attempt was successful, corresponding to the edge. Related It exhibits low lateral force, high coaxiality, and This is a recent timestamp. Among them, and As two independent edges coexisting, each carries its own unique characteristics. and This provides a basis for differentiated sampling for subsequent biased random walks.

[0052] In step 140, based on the multidirected graph, a biased random walk sampling strategy is adopted to generate a node sequence for graph embedding training; the random walk sampling strategy dynamically adjusts the sampling probability of neighboring nodes according to the data quality and timestamp freshness of the multi-source sensor information associated with each edge.

[0053] To generate a sequence of context nodes suitable for training graph embeddings, the robot performs a biased random walk on the constructed multidirected graph. Instead of using traditional uniform sampling, the random walk sampling strategy dynamically computes the values ​​starting from the current node. Transfer to its neighboring nodes The probability, which is determined by the connecting edge ( , The quality of the multi-source sensor information associated with it is determined by both the freshness of the timestamp.

[0054] As can be seen from the above description, for any slave node Pointing to node With directed edges, the robot has pre-stored two key attributes: the aggregated value of sensor data. , and timestamp aggregate value , The random walk strategy will take into account all factors. and The value of is: when the sensor data corresponding to a certain edge has higher quality and more recent timestamp, the probability of the neighboring node it points to being sampled is correspondingly increased; otherwise, it is decreased.

[0055] Through this mechanism, the random walk process can prioritize traversing state transition paths supported by high-quality, recent sensor data, thereby generating node sequences that better reflect effective operating patterns and providing highly discriminative contextual corpus for subsequent graph embedding training.

[0056] Specifically, in some implementations, the step "based on a multi-directed graph, using a biased random walk sampling strategy to generate a node sequence for graph embedding training" may include the following steps: (1) For the target node and its neighboring nodes, calculate the first transition probability based on the data quality of the multi-source sensor information associated with each edge and the second transition probability based on the freshness of the timestamp of the multi-source sensor information associated with each edge; the target node is any node in the multi-directed graph. (2) According to the preset weight parameters, the first transition probability and the second transition probability are fused to obtain the final transition probability; (3) Perform a biased random walk based on the final transition probability to generate a node sequence.

[0057] To achieve fine-grained traversal control over multidirected graphs, the robot employs a two-factor fusion biased random walk strategy. The core of this strategy lies in decoupling the factors influencing transition probabilities into two independent dimensions: "data quality" and "time freshness," modeling them separately, and then fusion them with weights.

[0058] For any target node in a multidirected graph (i.e., the node currently being traversed), the robot first examines all its outer neighboring nodes. Since multiple edges are allowed in the graph, each edge from... point to Each edge is associated with a set of multi-source sensor information, and two key attributes have been pre-extracted: the aggregated value of the sensor data. : and timestamp aggregate value .

[0059] Based on this, the first transition probability is only based on Calculations are used to measure the tendency to select nodes due to high data quality. The probability of the second transition; the probability of the second transition is based only on Calculations are used to measure the tendency to select nodes based on their newness of timestamp. The possibility.

[0060] Specifically, in some implementations, the step "for the target node and its neighboring nodes, calculate the first transition probability based on the data quality of the multi-source sensing information associated with each edge" may include the step of: for the target node and its neighboring nodes, calculating the first transition probability using the first transition probability formula; In one specific implementation, the first transition probability formula is expressed as: in, To start from the target node Neighboring nodes of the target node The first transition probability, For the target node Neighboring nodes of the target node The set of all multi-source sensor information related to each other, For the set of Quality score of the set of data For the target node The set of all neighboring nodes, For the target node and the neighboring nodes of the target node All multi-source sensor information associated with each other, For the target node and the neighboring nodes of the target node The first of all multi-source sensing information associated with each other Quality score of the set of data.

[0061] In some implementations, the step "for the target node and its neighboring nodes, calculate the second transition probability based on the freshness of the timestamps of the multi-source sensor information associated with each edge" may include the step of: for the target node and its neighboring nodes, calculating the second transition probability using the second transition probability formula.

[0062] In one specific implementation, the formula for the second transition probability is expressed as: in, For the target node Neighboring nodes of the target node The second transition probability, For the target node and the neighboring nodes of the target node The set of freshness scores for all timestamps between. For the target node and the neighboring nodes of the target node Between The timestamp freshness score corresponding to the multi-source sensor information. For the target node The set of all neighboring nodes, For the target node and the neighboring nodes of the target node The set of freshness scores for all timestamps between. For the target node and the neighboring nodes of the target node Between The timestamp freshness score corresponding to the multi-source sensor information.

[0063] The robot incorporates preset weight parameters to linearly fuse the first and second transition probabilities, yielding the final transition probability used for sampling. The preset weight parameters range from 0 to 1, and their specific values ​​can be configured according to the robot's task requirements. For example, in scenarios emphasizing operational stability, the preset weight parameters can be increased (prioritizing data quality). Conversely, in dynamic environment adaptation scenarios, the preset weight parameters can be decreased (focusing more on recent behavior).

[0064] Specifically, in some implementations, the step "fusing the first transition probability and the second transition probability according to the preset weight parameters to obtain the final transition probability" may include: fusing the first transition probability and the second transition probability according to the preset weight parameters and the weighted fusion formula to obtain the final transition probability.

[0065] In one specific implementation, the weighted fusion formula is expressed as: in, For preset weight parameters, The first transition probability, This represents the second transition probability.

[0066] The robot, based on the final transition probability, starts from the current node. Sample the next-hop node in the neighborhood according to probability. And repeat this process to generate a line of length . The node sequence is used as contextual corpus for subsequent graph embedding training.

[0067] It is worth noting that the biased walk strategy is applied to two sampling processes: the first is performed on the entire graph (including user nodes and item nodes) to generate the original sequence; the second is performed on a subgraph containing only robot operation state nodes (i.e., user nodes) to extract the higher-order operation semantic sequence.

[0068] Thus, the robot can flexibly balance the influence of high-quality historical data and recent behavioral trends, making the generated node sequences both reliable and timely, significantly improving the graph embedding's ability to represent effective operational patterns.

[0069] In step 150, the Skip-gram model is trained based on the node sequence to obtain a low-dimensional embedding vector for each node.

[0070] The Skip-gram model is a neural network model used to learn low-dimensional vector representations of discrete symbols. The robot treats a sequence of nodes generated through biased random walks as a structured corpus of "sentences," where each node (such as a robot's operational state or an environmental object) is considered a "word." The Skip-gram model takes this node sequence as input and automatically learns a low-dimensional dense vector representation (i.e., an embedding vector) for each node by maximizing the co-occurrence probability of the central node with its context neighbors.

[0071] The robot is trained on a Skip-gram model to analyze the node sequence. The core objective of the Skip-gram model is to predict a node itself using its context neighbors, or equivalently, to predict its neighboring nodes using the central node.

[0072] During training, the Skip-gram model learns a mapping function that encodes each discrete node in the graph (e.g., operation state nodes such as "approaching target" or "stable grasp", or item nodes such as "pin" or "hole") into a fixed-dimensional low-dimensional real-valued vector (i.e., embedding vector), so that nodes with similar contextual behaviors in the original graph are close to each other in the embedding space.

[0073] Embedding vectors can capture the structural role and interaction semantics of nodes in multidirected graphs. For example, operational state nodes that frequently appear in similar force-visual contexts will have their embedding vectors close to each other; robot states that frequently interact with the same type of workpiece will form clusters in the embedding space.

[0074] It is worth noting that this invention performs two independent Skip-gram model trainings to achieve multi-scale embedding fusion. Specifically, it trains on node sequences generated on the entire graph (including user nodes and item nodes) to obtain embeddings that reflect the joint interaction semantics of "robot-environment". It also trains on sequences extracted from a subgraph containing only user nodes (i.e., robot operation state nodes) to obtain high-order operation embeddings that focus on the evolution of the robot's own behavior. The fusion of these two forms a multi-scale low-dimensional embedding representation for each robot operation state node, which can be used for downstream tasks (such as operation trajectory optimization, anomaly detection, or policy generation).

[0075] For example, assuming that the node "Contact Adjustment_A" appears after "Approaching the Target" and before "Insertion" in multiple successful insertion tasks, and is accompanied by high-quality force and visual data, its embedding vector will be highly correlated with the "Successful Operation Mode" in the vector space; while the nodes corresponding to failed attempts are suppressed during the sampling phase, and have less impact on the embedding, thereby improving the robustness of the embedding.

[0076] Specifically, in some implementations, the step "training the Skip-gram model based on the node sequence to obtain a low-dimensional embedding vector for each node" may include the following steps: (1) Extract a subsequence containing only robot operation state nodes from the node sequence as a sampling sequence; (2) Construct the first corpus and the second corpus based on the node sequence and the sampled sequence, respectively; (3) Train the first Skip-gram model based on the first corpus to obtain the first embedding vector of each node; (4) Train the second Skip-gram model based on the second corpus to obtain the second embedding vector of each node; (5) For each robot operation state node, the first embedding vector and the second embedding vector are concatenated or weighted and fused to obtain the low-dimensional embedding vector of the node.

[0077] The robot generates the original node sequence on the multidirected graph using a biased random walk strategy. As described above, the graph contains two types of nodes: user nodes (i.e., robot operation state nodes, such as "approaching the target", "contact adjustment", etc.) and object nodes (i.e., task objects in the environment, such as "screws", "holes", etc.).

[0078] Based on this, the robot filters the original node sequence, removing all item nodes and retaining only user nodes, thus obtaining a subsequence containing only robot operation state nodes. This subsequence is defined as a sampling sequence, used to capture high-order patterns of the robot's own behavioral evolution.

[0079] The robot uses the original node sequence as the first corpus and the sampled sequence as the second corpus. The first and second corpora reflect interaction contexts of different granularities. That is, the first corpus retains the complete "robot-environment-robot" interaction chain, while the second corpus focuses on the transition patterns between robot operation states.

[0080] The first Skip-gram model takes the first corpus as input, learns the embedding representation of all nodes (including user nodes and item nodes) in the graph, and outputs the first embedding vector of each node; the second Skip-gram model takes the second corpus as input, learns only the embedding representation of user nodes (i.e. robot operation status nodes), and outputs the second embedding vector of each user node.

[0081] For each robot operation state node (i.e., user node), the robot fuses the embedding vectors obtained from the two models. The fusion methods may include vector concatenation (connecting the first and second embedding vectors end to end to form a higher-dimensional but more information-rich representation) and weighted fusion (linearly combining the two vectors with preset weights).

[0082] In some implementations, the robot correction method further includes: training a first Skip-gram model and a second Skip-gram model respectively by optimizing an objective function to obtain a first embedding vector and a second embedding vector.

[0083] In one specific implementation, the objective function is expressed as: in, For the first Scale corpus, It is an exponential function. For nodes , For nodes , For nodes Embedded vector, For nodes Embedded vectors.

[0084] The fused vector is the final low-dimensional embedding vector of the robot's operation state node. It simultaneously contains cross-modal environment interaction semantics (from the first embedding) and pure operation behavior evolution rules (from the second embedding), thus providing a highly discriminative feature representation for subsequent operation accuracy optimization (such as trajectory correction, failure prediction, and policy generation).

[0085] In step 160, the robot's motion trajectory or operation strategy is corrected based on the low-dimensional embedding vector.

[0086] When a robot performs a new task and enters a certain operational state (e.g., "approaching the target"), the robot calculates the embedding vector of that state in real time and retrieves the nearest historical successful operational state in the embedding space (e.g., by cosine similarity or Euclidean distance). If the current embedding deviates significantly from the center of the successful cluster, an operational deviation is determined (e.g., angular skew, excessive speed), triggering a trajectory correction mechanism. For example, adjusting the pose of the end effector, reducing the feed rate, and / or introducing a fine-tuning force control strategy.

[0087] This process leverages the highly compressed representation capability of embedded vectors for multimodal sensing contexts, avoiding the complexity of directly processing raw high-dimensional sensing data.

[0088] During the decision-making phase, the robot can input the embedding vector of its current state into a lightweight policy network (such as a classifier or regressor). This network has already learned the mapping relationship between "embedding - optimal next action" during the training phase. For example, if the embedding is close to the "successful insertion" cluster, it continues to move forward; if the embedding is close to the "stuck and failed" region, it backs down and realigns.

[0089] For example, in the "pin insertion" task, if the embedding vector of the current "aligned" state is highly similar to the embedding that failed in the past due to excessive lateral force, the robot can reduce the axial thrust in advance and activate the visual fine-tuning module to avoid repeated failures.

[0090] Please see Figure 2 , Figure 2 This illustration shows a schematic diagram of a robot calibration device according to an embodiment of this application. The robot body and its environment are equipped with multiple heterogeneous sensors. The robot calibration device 200 includes: a data acquisition module 210, a preprocessing module 220, a construction module 230, a generation module 240, a training module 250, and an execution module 260. Specifically: The data acquisition module 210 is used to acquire multidimensional raw sensing data through multiple heterogeneous sensors during the robot's task execution. Preprocessing module 220 is used to perform spatiotemporal calibration and noise filtering on multidimensional raw sensor data to obtain synchronized and aligned multi-source sensor information; The construction module 230 is used to construct a multi-directed graph based on multi-source sensor information. Its nodes represent robot operation states, edges represent state transition instances, and each edge is accompanied by its corresponding multi-source sensor information. The generation module 240 is used to generate a node sequence for graph embedding training based on a multi-directed graph and a biased random walk sampling strategy. The random walk sampling strategy dynamically adjusts the sampling probability of neighboring nodes according to the data quality and timestamp freshness of the multi-source sensor information associated with each edge. Training module 250 is used to train the Skip-gram model based on the node sequence to obtain a low-dimensional embedding vector for each node. The execution module 260 is used to correct the robot's motion trajectory or operation strategy based on low-dimensional embedding vectors.

[0091] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described device and module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0092] In the several embodiments provided in this application, the coupling or direct coupling or communication connection between the modules shown or discussed may be an indirect coupling or communication connection through some interface, device or module, and may be electrical, mechanical or other forms.

[0093] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0094] Please see Figure 3 , Figure 3 The diagram shows a schematic of the structure of a robot provided in an embodiment of this application. The robot 300 in this application may include one or more of the following components: a processor 310, a memory 320, and one or more application programs. The one or more application programs may be stored in the memory 320 and configured to be executed by one or more processors 310. The one or more programs are configured to perform the robot correction method as described in the foregoing method embodiments.

[0095] The processor 310 may include one or more processing cores. The processor 310 connects to various parts within the robot 300 using various interfaces and lines, and performs various functions and processes data of the robot 300 by running or executing instructions, programs, code sets, or instruction sets stored in the memory 320, and by calling data stored in the memory 320. Optionally, the processor 310 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 310 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 310 and may be implemented separately using a communication chip.

[0096] The memory 320 may include random access memory (RAM) or read-only memory (ROM). The memory 320 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 320 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing the various method embodiments described below, etc. The data storage area may also store data created by the robot 300 during use.

[0097] Please see Figure 4 , Figure 4 The diagram shows a computer-readable storage medium 400 provided in an embodiment of this application. The computer-readable storage medium 400 stores program code, which can be called by a processor to execute the robot correction method described in the above method embodiment.

[0098] The computer-readable storage medium 400 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 400 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 400 has storage space for program code 410 that performs any of the method steps described above. This program code can be read from or written to one or more computer program devices. The program code 410 may be compressed, for example, in a suitable form.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A robot calibration method, characterized in that, The robot body and its environment are equipped with multiple heterogeneous sensors, and the method includes: During the robot's task execution, multidimensional raw sensing data is collected through the multiple heterogeneous sensors; Spatiotemporal calibration and noise filtering are performed on the multidimensional raw sensor data to obtain synchronized and aligned multi-source sensor information. Based on the multi-source sensing information, a multi-directed graph is constructed, where nodes represent robot operation states, edges represent state transition instances, and each edge is accompanied by its corresponding multi-source sensing information. Based on the multidirected graph, a biased random walk sampling strategy is adopted to generate a node sequence for graph embedding training; the random walk sampling strategy dynamically adjusts the sampling probability of neighboring nodes according to the data quality and timestamp freshness of the multi-source sensor information associated with each edge. The Skip-gram model is trained based on the node sequence to obtain a low-dimensional embedding vector for each node; The robot's motion trajectory or operation strategy is corrected based on the low-dimensional embedding vector.

2. The robot calibration method according to claim 1, characterized in that, The step of generating a node sequence for graph embedding training based on the multidirected graph using a biased random walk sampling strategy includes: For the target node and its neighboring nodes, calculate a first transition probability based on the data quality of the multi-source sensor information associated with each edge and a second transition probability based on the freshness of the timestamps of the multi-source sensor information associated with each edge; the target node is any node in the multi-directed graph. According to the preset weight parameters, the first transition probability and the second transition probability are fused to obtain the final transition probability; A biased random walk is performed based on the final transition probability to generate the node sequence.

3. The robot calibration method according to claim 2, characterized in that, The step, for the target node and its neighboring nodes, calculates a first transition probability of data quality based on the multi-source sensor information associated with each edge, including: For the target node and its neighboring nodes, the first transition probability is calculated using the first transition probability formula. The first transition probability formula is expressed as: in, To start from the target node Neighboring nodes of the target node The first transition probability, For the target node Neighboring nodes of the target node The set of all multi-source sensor information related to each other, For the set of Quality score of the set of data For the target node The set of all neighboring nodes, For the target node and the neighboring nodes of the target node All multi-source sensor information associated with each other, For the target node and the neighboring nodes of the target node The first of all multi-source sensing information associated with each other Quality score of the set of data.

4. The robot calibration method according to claim 2, characterized in that, The calculation of the second transition probability based on the freshness of the timestamps of the multi-source sensor information associated with each edge, for the target node and its neighboring nodes, includes: For the target node and its neighboring nodes, the second transition probability is calculated using the second transition probability formula. The second transition probability formula is expressed as: in, For the target node Neighboring nodes of the target node The second transition probability, For the target node and the neighboring nodes of the target node The set of freshness scores for all timestamps between. For the target node and the neighboring nodes of the target node Between The timestamp freshness score corresponding to the multi-source sensor information. For the target node The set of all neighboring nodes, For the target node and the neighboring nodes of the target node The set of freshness scores for all timestamps between. For the target node and the neighboring nodes of the target node Between The timestamp freshness score corresponding to the multi-source sensor information.

5. The robot calibration method according to claim 2, characterized in that, The step of fusing the first transition probability and the second transition probability according to preset weight parameters to obtain the final transition probability includes: According to the preset weight parameters and weighted fusion formula, the first transition probability and the second transition probability are fused to obtain the final transition probability; The weighted fusion formula is expressed as follows: in, The preset weight parameters, The first transition probability, This represents the second transition probability.

6. The robot calibration method according to claim 1, characterized in that, The step of training the Skip-gram model based on the node sequence to obtain a low-dimensional embedding vector for each node includes: Extract a subsequence containing only robot operation state nodes from the node sequence and use it as a sampling sequence; A first corpus and a second corpus are constructed based on the node sequence and the sampling sequence, respectively. The first Skip-gram model is trained based on the first corpus to obtain the first embedding vector of each node; The second Skip-gram model is trained based on the second corpus to obtain the second embedding vector of each node; For each robot operation state node, the corresponding first embedding vector and the second embedding vector are concatenated or weighted and fused to obtain the low-dimensional embedding vector of that node.

7. The robot calibration method according to claim 6, characterized in that, The method further includes: The first Skip-gram model and the second Skip-gram model are trained by optimizing the objective function to obtain the first embedding vector and the second embedding vector. The objective function is expressed as: in, For the first Scale corpus, It is an exponential function. For nodes , For nodes , For nodes Embedded vector, For nodes Embedded vectors.

8. A robot calibration device, characterized in that, The robot body and its environment are equipped with multiple heterogeneous sensors, and the device includes: The data acquisition module is used to acquire multidimensional raw sensor data through the multiple heterogeneous sensors during the robot's task execution. The preprocessing module is used to perform spatiotemporal calibration and noise filtering on the multidimensional raw sensor data to obtain synchronized and aligned multi-source sensor information. The construction module is used to construct a multi-directed graph based on the multi-source sensing information, where nodes represent robot operation states, edges represent state transition instances, and each edge is accompanied by its corresponding multi-source sensing information. The generation module is used to generate a node sequence for graph embedding training based on the multi-directed graph using a biased random walk sampling strategy; the random walk sampling strategy dynamically adjusts the sampling probability of neighboring nodes according to the data quality and timestamp freshness of the multi-source sensor information associated with each edge. The training module is used to train the Skip-gram model based on the node sequence to obtain a low-dimensional embedding vector for each node. An execution module is used to correct the robot's motion trajectory or operation strategy based on the low-dimensional embedding vector.

9. A robot, characterized in that, include: One or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications being configured to perform the robot correction method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be called by a processor to execute the robot correction method as described in any one of claims 1-7.

Images