Robot key action detection method and device and computer storage medium

Abstract

The application provides a robot key action detection method and device and a computer storage medium. By introducing a posterior task result description quantity as a constraint condition for training of a conditional action distribution model, the model is no longer based on historical conditions to predict actions, but learns to determine the conditional action distribution corresponding to an action sequence under given historical conditions and the posterior task result description quantity. The local action modal in the conditional action distribution is used to distinguish different action schemes that can be replaced with each other under the same task result. Further, the width of the actionable domain in the local region where the current demonstration action belongs is determined based on the local action modal, the width represents the range of allowed changes of the action under the premise of maintaining the same task result, the narrower the width, the stricter the accuracy constraint of the task on the action, and the key action interval is determined based on the local actionable domain width. The key action can be detected from the demonstration data without relying on any pre-trained strategy model or artificial rules.

Description

Technical Field

[0001] This invention relates to the fields of robot control and artificial intelligence technology, and in particular to a method, device and computer storage medium for detecting key robot movements. Background Technology

[0002] In the evolution of robotics and embodied intelligence, the ability of robots to perform general operations across scenarios and tasks is receiving increasing attention. Imitation learning, as the mainstream paradigm for achieving general operations, uses deep learning algorithms to enable robots to learn operational strategies from massive demonstration datasets. However, demonstration datasets often contain a large number of non-critical actions, such as pauses, jitters, or random movements. Teaching robots to learn these actions not only reduces training efficiency but also leads to poor final performance due to noisy signals.

[0003] Existing key action detection methods mainly fall into two categories. One category determines the degree of criticality based on the prediction error of the robot's policy model. However, the training objective of the policy model is only to fit the observed actions, rather than to fully represent all alternative actions under given task conditions. This results in the evaluation results being severely limited by the capabilities of a specific model, easily missing a large number of potential key actions, and having low accuracy. The other category relies on manually set rules for screening, such as classifying key actions by gripper state or acceleration threshold. This type of method requires redefining the rules for new scenarios, new tasks, or new hardware, lacking versatility and automation capabilities.

[0004] The common drawback of the above methods is that they all fail to make full use of the distribution characteristics inherent in the demonstration data itself. The detection results are difficult to truly reflect the task's constraints on action accuracy, which in turn leads to insufficient focus on the truly critical actions in subsequent training. Summary of the Invention

[0005] In view of this, the present invention provides a method, device and computer storage medium for detecting key robot movements, so as to improve the accuracy and versatility of key movement detection.

[0006] The first aspect of this invention provides a method for detecting key robot actions, comprising:

[0007] Obtain the robot's demonstration trajectory; wherein the demonstration trajectory includes state information and action information at multiple moments;

[0008] For each anchor point moment, construct historical conditions, posterior task result descriptors, and action sequences following the anchor point moment; wherein, the historical conditions are constructed based on state information and action information before the anchor point moment, and the posterior task result descriptors are constructed based on state information following the anchor point moment.

[0009] A conditional action distribution model is trained using a sample group constructed with the historical conditions, the posterior task result descriptor, and the action sequence to obtain a trained conditional action distribution model; wherein, the trained conditional action distribution model determines the conditional action distribution corresponding to the action sequence based on the historical conditions and the posterior task result descriptor; the conditional action distribution includes multiple local action modalities;

[0010] Based on the conditional action distribution, determine the local action modality to which the current demonstration action sequence belongs;

[0011] Based on the local action modality to which the current demonstration action sequence belongs, determine the width of the local action domain of the local region where the current demonstration action sequence is located;

[0012] The criticality of the anchor point moment is determined based on the width of the local action domain, and the critical action range is determined based on the criticality.

[0013] Optionally, training the conditional action distribution model using the sample group constructed with the historical conditions, the posterior task result descriptor, and the action sequence to obtain the trained conditional action distribution model includes:

[0014] The historical conditions, the posterior task result descriptor, and the action sequence constitute a sample group;

[0015] The bidirectional occlusion Transformer model is used to occlude the target action sequence in the sample group and the action data in the adjacent isolation area of ​​the target action sequence to obtain the occluded data.

[0016] The conditional action distribution model is trained based on the occlusion-processed data to obtain the trained conditional action distribution model.

[0017] Optionally, determining the local action modality to which the current demonstration action sequence belongs based on the conditional action distribution includes:

[0018] Obtain the current demonstration action sequence;

[0019] Calculate the posterior responsibility of the current demonstration action sequence relative to each local action modality in the conditional action distribution;

[0020] The local action modality corresponding to the largest posterior responsibility is selected as the local action modality to which the current demonstration action sequence belongs.

[0021] Optionally, determining the width of the local action domain of the local region where the current demonstration action sequence is located, based on the local action modality to which the current demonstration action sequence belongs, includes:

[0022] Obtain the direction-related covariance matrix within the local action modality to which the current demonstration action sequence belongs;

[0023] The width of the local movable domain of the local region is determined based on the determinant of the direction-related covariance matrix; wherein, the smaller the determinant value of the direction-related covariance matrix, the narrower the corresponding local movable domain width.

[0024] Optionally, determining the criticality of the anchor point moment based on the width of the local movable action domain, and determining the critical action interval based on the criticality, includes:

[0025] The width of the local movable action domain is standardized to obtain the criticality of the anchor point moment; wherein the criticality is negatively correlated with the width of the local movable action domain.

[0026] The criticality level is then filtered and smoothed to obtain a smoothed criticality level.

[0027] The criticality of the smoothed process is divided into critical moments and non-critical moments based on a preset threshold.

[0028] Merge consecutive key moments within the same round into candidate intervals;

[0029] The candidate intervals are filtered to obtain key action intervals; wherein, the filtering process includes deleting candidate intervals whose duration is less than a preset duration threshold, and merging adjacent candidate intervals whose interval is less than a preset interval threshold to obtain the key action intervals.

[0030] Optionally, after determining the criticality of the anchor point moment based on the width of the local movable action domain, the method further includes:

[0031] Training weights for the anchor point are generated based on the criticality level; wherein the training weights are positively correlated with the criticality level.

[0032] Optionally, after generating the training weights for the anchor point time based on the criticality, the method further includes:

[0033] Save the demonstration round identifier, frame index, criticality level, critical action interval identifier, and training weights corresponding to each anchor point as independent metadata.

[0034] Optionally, after saving the demonstration round identifier, frame index, criticality level, key action interval identifier, and training weights corresponding to each anchor point as independent metadata, the method further includes:

[0035] Obtain the independent metadata;

[0036] The target robot policy model is trained based on the training weights in the independent metadata; wherein the network parameters of the target robot policy model are independent of the network parameters of the width estimator used to determine the width of the local action domain.

[0037] A second aspect of the present invention provides a robot key motion detection device, comprising:

[0038] A trajectory acquisition unit is used to acquire the robot's demonstration trajectory; wherein, the demonstration trajectory includes state information and action information at multiple moments;

[0039] A data construction unit is used to construct historical conditions, posterior task result descriptors, and action sequences after the anchor point time; wherein, the historical conditions are constructed based on state information and action information before the anchor point time, and the posterior task result descriptors are constructed based on state information after the anchor point time.

[0040] A distribution training unit is used to train a conditional action distribution model using a sample group constructed with the historical conditions, the posterior task result descriptor, and the action sequence, to obtain a trained conditional action distribution model; wherein, the trained conditional action distribution model determines the conditional action distribution corresponding to the action sequence based on the historical conditions and the posterior task result descriptor; the conditional action distribution includes multiple local action modalities.

[0041] The modality determination unit is used to determine the local action modality to which the current demonstration action sequence belongs based on the conditional action distribution.

[0042] The width determination unit is used to determine the width of the local action domain of the local region where the current demonstration action sequence is located, based on the local action modality to which the current demonstration action sequence belongs.

[0043] The critical action interval determination unit is used to determine the criticality of the anchor point moment based on the width of the local movable action domain, and to determine the critical action interval according to the criticality.

[0044] Optionally, the distributed training unit includes:

[0045] A sample group constituting unit is used to form a sample group from the historical conditions, the posterior task result descriptor, and the action sequence.

[0046] The occlusion processing unit is used to perform occlusion processing on the target action sequence in the sample group and the action data in the adjacent isolation area of ​​the target action sequence using a two-way occlusion Transformer model to obtain the occlusion processed data.

[0047] The Conditional Action Distribution Model Training Unit is used to train the Conditional Action Distribution Model based on the occlusion-processed data to obtain the trained Conditional Action Distribution Model.

[0048] Optionally, the mode determination unit includes:

[0049] The current demonstration action sequence acquisition unit is used to acquire the current demonstration action sequence;

[0050] The posterior responsibility calculation unit is used to calculate the posterior responsibility of the current demonstration action sequence relative to each local action modality in the conditional action distribution;

[0051] The selection unit is used to select the local action modality corresponding to the largest posterior responsibility as the local action modality to which the current demonstration action sequence belongs.

[0052] Optionally, the width determining unit includes:

[0053] The directional covariance matrix acquisition unit is used to acquire the directional covariance matrix within the local action modality to which the current demonstration action sequence belongs;

[0054] A local movable action domain width determination unit is used to determine the local movable action domain width of the local region based on the determinant of the direction correlation covariance matrix; wherein, the smaller the determinant value of the direction correlation covariance matrix, the narrower the corresponding local movable action domain width.

[0055] Optionally, the key action range determination unit includes:

[0056] A standardization processing unit is used to standardize the width of the local movable action domain to obtain the criticality of the anchor point moment; wherein the criticality is negatively correlated with the width of the local movable action domain.

[0057] A filtering and smoothing processing unit is used to perform filtering and smoothing processing on the criticality level to obtain a smoothed criticality level.

[0058] A segmentation unit is used to divide the criticality of the smoothed process into critical moments and non-critical moments according to a preset threshold.

[0059] The merging unit is used to merge consecutive key moments within the same round into candidate intervals;

[0060] A filtering unit is used to filter the candidate intervals to obtain key action intervals; wherein, the filtering process includes deleting candidate intervals whose duration is less than a preset duration threshold, and merging adjacent candidate intervals whose interval is less than a preset interval threshold to obtain the key action intervals.

[0061] Optionally, the robot key motion detection device further includes:

[0062] A training weight generation unit is used to generate training weights for the anchor point time based on the criticality level; wherein the training weights are positively correlated with the criticality level.

[0063] Optionally, the robot key motion detection device further includes:

[0064] The storage unit is used to save the demonstration round identifier, frame index, criticality, critical action interval identifier, and training weights corresponding to each anchor point as independent metadata.

[0065] Optionally, the robot key motion detection device further includes:

[0066] An independent metadata acquisition unit is used to acquire the independent metadata;

[0067] A robot policy model training unit is used to train a target robot policy model based on the training weights in the independent metadata; wherein the network parameters of the target robot policy model are independent of the network parameters of the width estimator used to determine the width of the local action domain.

[0068] A third aspect of the present invention provides an electronic device, comprising:

[0069] One or more processors;

[0070] A storage device on which one or more programs are stored;

[0071] When the one or more programs are executed by the one or more processors, the one or more processors implement the robot key motion detection method as described in any one of the first aspects.

[0072] A fourth aspect of the present invention provides a computer storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the robot key motion detection method as described in any one of the first aspects.

[0073] As can be seen from the above scheme, the present invention provides a method, device, and computer storage medium for detecting key robot actions. By introducing a posterior task result descriptor as a constraint condition during the training of the conditional action distribution model, the model no longer predicts actions solely based on historical conditions. Instead, it learns to determine the conditional action distribution corresponding to the action sequence given historical conditions and the posterior task result descriptor. Based on this, it uses local action modalities in the conditional action distribution to distinguish different action schemes that can be substituted for each other under the same task result. Furthermore, it determines the width of the feasible action domain of the local region to which the current demonstration action belongs based on the local action modality. This width directly represents the range of allowed variation of the action while maintaining the same task result; the narrower the width, the stricter the task's precision constraint on the action. Therefore, key actions can be automatically detected from demonstration data without relying on any pre-trained strategy model or manual rules. The detected key actions directly address the core aspects of task success or failure, effectively filtering out non-critical signals such as pauses and jitter. Attached Figure Description

[0074] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0075] Figure 1 A flowchart of a robot key motion detection method provided in an embodiment of the present invention;

[0076] Figure 2 A flowchart of a training conditional action distribution model method provided in another embodiment of the present invention;

[0077] Figure 3 A flowchart of a method for determining local action modes provided in another embodiment of the present invention;

[0078] Figure 4 A flowchart illustrating a method for determining key action intervals according to another embodiment of the present invention;

[0079] Figure 5 A schematic diagram of a robot key motion detection device provided in another embodiment of the present invention;

[0080] Figure 6 This is a schematic diagram of an electronic device for implementing a method for detecting key robot movements, provided as another embodiment of the present invention. Detailed Implementation

[0081] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0082] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0083] It should be noted that the information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this invention are all information and data authorized by the user or fully authorized by all parties.

[0084] It should be noted that the concepts of "first" and "second" mentioned in this invention are only used to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0085] It should be noted that the terms "a" and "a plurality of" used in this invention are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0086] This invention provides a method for detecting key robot actions, such as... Figure 1 As shown, the specific steps include:

[0087] S101. Obtain the robot demonstration trajectory.

[0088] The demonstration trajectory includes state and motion information at multiple points in time. The state information includes observation data and robot state data, such as robot end-effector pose, robot joint angles, and camera images. The motion information includes robot motion data, such as the joint angle sequence to be executed and the tracked end-effector pose trajectory.

[0089] In the practical application of this invention, the demonstration trajectory can be derived from a single successful demonstration data point collected for a single task, or from a massive demonstration dataset containing multiple scenarios and multiple tasks; no limitation is made here.

[0090] S102. Construct historical conditions, posterior task result descriptions, and action sequences after the anchor point time for the anchor point time.

[0091] Among them, the historical conditions are constructed based on the state and action information before the anchor point time, and the posterior task result description is constructed based on the state information after the anchor point time.

[0092] It should be noted that the anchor point time is an arbitrarily selected time point in the demonstration trajectory, denoted as t.

[0093] In this embodiment of the invention, each moment in the demonstration trajectory can be traversed with a preset time step, or only moments within a specific time period can be selected as anchor moments according to the characteristics of the task; no limitation is made here.

[0094] Among them, historical conditions The context used to represent the decision context before the anchor point can be constructed by encoding the state and action information of a preset number of moments before the anchor point, without any limitation here.

[0095] For example, in one embodiment of the present invention, historical conditions can be represented as :

[0096] ;

[0097] in, express Robot observation data from time point A to the current time. express The robot's state data from time point A to the current time point; Enc is a trainable historical conditional encoder. Posterior task result descriptor. This is used to characterize the task result achieved by the action execution after the anchor point moment. In the embodiments of the present invention, It can be constructed based on the state information within the target time period after the anchor point time. The target time period includes a continuous segment of time starting from the anchor point time.

[0098] For example, in one implementation, the posterior task outcome descriptor... It can be represented as:

[0099] ;

[0100] in, It is the future arrive Robot observation data at any given time It is the future arrive The robot's state data at any given time, Result, is a trainable posterior task result encoder. In practical applications of this invention, the posterior task result description may include label information indicating task success or failure, quantitative indicators of task completion progress, and more granular descriptions of state changes. The specific form and content depend on the task type and scenario requirements, and those skilled in the art can choose according to actual applications; no limitations are imposed here.

[0101] It should be noted that the descriptive quantity of the posterior task results The construction requires the use of information after the anchor point time, which is consistent with the core idea of ​​the posterior of this invention, that is, to infer the set of possible actions based on the knowledge of the action execution result, rather than predicting the future based solely on historical information.

[0102] Among them, action sequence The sequence is a set of action data for a predetermined number of time intervals H following the anchor point. In this embodiment of the invention, the action sequence... It can be represented as:

[0103] ;in, Let t be the robot's action at time t. Used to represent robot actions within a continuous time segment.

[0104] S103. The conditional action distribution model is trained using the sample group constructed with historical conditions, posterior task result descriptors and action sequences to obtain the trained conditional action distribution model.

[0105] The trained conditional action distribution model determines the conditional action distribution corresponding to the action sequence based on historical conditions and posterior task result descriptions; the conditional action distribution contains multiple local action modalities.

[0106] It should be noted that the target of the conditional action distribution model learning in this step is the conditional probability distribution. That is, under given historical conditions and posterior task result descriptive quantity Under the condition of action sequence The probability distribution describes which sequences of actions are feasible and interchangeable in the current historical context to achieve a given task outcome.

[0107] In this embodiment of the invention, the sample group consists of It consists of triplets, among which and As model input This serves as a supervisory signal for model learning. Through training with a large amount of sample data at anchor times, the model can learn to perform training under the same historical conditions. and the same posterior outcome conditions Below, action sequence Multiple local action modalities may be presented.

[0108] Specifically, the conditional action distribution model can employ a Gaussian mixture model or other network structures capable of outputting multimodal distributions. In this embodiment of the invention, the model outputs a conditional distribution containing K local action modes, where K is a pre-defined number of modes, or can be automatically determined by the model; no limitation is made here. Each local action mode can represent an effective action scheme under the same outcome conditions.

[0109] Optionally, in another embodiment of the present invention, one implementation of step S103 is as follows: Figure 2 As shown, it includes:

[0110] S201. The historical conditions, posterior task result descriptors, and action sequences are combined to form a sample group.

[0111] Specifically, for each anchor point time t, construct the following according to the method in S102: , and Then, the three are linked together to form a complete training sample. When processing the entire demonstration trajectory or the entire dataset, a large number of training samples can be obtained, forming sample groups for training the conditional action distribution model.

[0112] S202. Use the bidirectional occlusion Transformer model to occlude the target action sequence and the action data in the adjacent isolation area of ​​the target action sequence in the sample group to obtain the occluded data.

[0113] It should be noted that, when training the conditional action distribution model using sample groups, to prevent the model from using information from the target action sequence itself and causing learning failure, this invention employs a bidirectional occlusion Transformer mechanism to occlude the target action sequence and the action data within the adjacent isolation zone. Occlusion refers to hiding the action data at specific locations during model computation, preventing the model from directly obtaining information from these locations.

[0114] Specifically, when calculating attention weights, the bidirectional occlusion Transformer model masks the action data corresponding to the time of the target action sequence and the time of the adjacent isolation zone, so that the model can only predict the conditional action distribution based on historical conditions. and posterior task result descriptive quantity Make inferences.

[0115] The range of adjacent quarantine zones is pre-defined and is not limited here.

[0116] It should be noted that after the occlusion process, the model learns to infer the set of possible actions based on historical information and the results of action execution during training, rather than simply copying the action data in the input, which effectively avoids the problem of information leakage during training.

[0117] S203. Train the conditional action distribution model based on the data after occlusion processing to obtain the trained conditional action distribution model.

[0118] After masking, the masked data is used to train a conditional action distribution model. The model learns from the data by minimizing the negative log-likelihood loss function. arrive The mapping relationship. After training, the conditional action distribution model can distinguish multiple local action modalities under the same historical conditions and the same posterior task result descriptor constraints. That is, it can identify the differences between different action schemes under the same result conditions and the allowable range of variation within each action scheme.

[0119] S104. Based on the conditional action distribution, determine the local action mode to which the current demonstration action sequence belongs.

[0120] Specifically, in this step, for the current anchor point moment in the demonstration trajectory, the local action mode to which the current demonstration action sequence belongs is determined based on the conditional action distribution obtained from training in step S103. Current demonstration action sequence This refers to the sequence of actions actually executed after time t at the anchor point in the demonstration trajectory.

[0121] Optionally, in another embodiment of the present invention, one implementation of step S104 is as follows: Figure 3 As shown, it includes:

[0122] S301. Obtain the current demonstration action sequence.

[0123] Specifically, actual motion data at a preset number of times after the anchor point time t is extracted from the demonstration trajectory to form the current demonstration motion sequence. .

[0124] S302. Calculate the posterior responsibility of the current demonstration action sequence relative to each local action modality in the conditional action distribution.

[0125] In this embodiment of the invention, when the conditional action distribution contains K local action modes, for the current demonstration action sequence Calculate its posterior responsibility for the k-th mode. The specific calculation formula can be:

[0126] ;in , It refers to the conditional action probability predicted by the k-th modality.

[0127] S303. Select the local action modality corresponding to the largest posterior responsibility as the local action modality to which the current demonstration action sequence belongs.

[0128] Specifically, let , will the Each modality is determined as the local action modality to which the current demonstration action sequence belongs. This modality represents the action plan that best matches the currently executed action sequence given historical conditions and task results.

[0129] S105. Based on the local action modality to which the current demonstration action sequence belongs, determine the width of the local action domain of the local region where the current demonstration action sequence is located.

[0130] In this embodiment of the invention, each local action modality has a corresponding covariance structure, which reflects the relationship between the dimensions of the action data under the action scheme. The covariance structure can be obtained directly from the trained conditional action distribution model, or it can be obtained by statistical analysis of training samples belonging to the same modality; no limitation is made here.

[0131] Optionally, in another embodiment of the present invention, one implementation of step S105 includes:

[0132] Obtain the direction-dependent covariance matrix within the local action modality to which the current demonstration action sequence belongs, and determine the width of the local action domain of the local region based on the determinant of the direction-dependent covariance matrix.

[0133] Specifically, let the first Each modality is a local action modality to which the current demonstration action sequence belongs. The direction-related covariance matrix within this modality is then obtained. The directional covariance matrix not only contains variance information in each dimension of the action sequence, but also covariance information between different dimensions, thus reflecting the anisotropic characteristics of action changes in different directions.

[0134] In the practical application of this invention, it can be based on Define the local action change domain :

[0135] ;in, This is a domain boundary parameter used to define the width of the domain. For feasible action variations, the action sequences in the application are derived from the dataset.

[0136] In this embodiment of the invention, constants independent of time t are ignored, and the size of the local feasible action domain is represented by the following logarithmic width. :

[0137] ;

[0138] Among them, the smaller the determinant value of the direction-related covariance matrix, the larger the corresponding local actionable domain. The narrower the width, the smaller the range of action variations that can maintain the same task result under the action plan, and the more critical that moment is. The eigenvalues ​​of Σ represent the width in different action directions, so that the domain width not only reflects the overall size but also preserves directional correlation.

[0139] It should be noted that the present invention solves the problem of estimating the distance interference width between different effective action schemes in the prior art by assigning the current action to a specific local action mode and calculating the width of the actionable domain based on the covariance matrix within that mode.

[0140] Specifically, since different local action modes correspond to completely different action schemes, the distance between these schemes should not be included in the allowable variation range of a particular action scheme. This invention only calculates the covariance structure within the mode to which the current action belongs, thereby accurately reflecting the task accuracy constraints of the action scheme itself.

[0141] S106. Determine the criticality of the anchor point moment based on the width of the local action domain, and determine the critical action range according to the criticality.

[0142] In this context, a smaller local action domain width indicates a smaller action tolerance and higher precision requirement at the current action location at the anchor point, meaning that the moment is more critical. Based on this principle, this invention determines the criticality of the anchor point moment and further identifies the critical action range.

[0143] Optionally, in another embodiment of the present invention, one implementation of step S106 is as follows: Figure 4 As shown, it includes:

[0144] S401. Standardize the width of the local actionable domain to obtain the criticality of the anchor point moment.

[0145] The degree of criticality is negatively correlated with the width of the local action domain. The narrower the width of the local action domain, the higher the degree of criticality.

[0146] In embodiments of the invention, the width of the local actionable domain can be standardized by round, task, or dataset range. For example, the degree of criticality... It can be calculated in the following ways:

[0147] ;in, for the median of This represents the absolute deviation of the median. It is a small real number to increase computational stability; 1.4826 is a commonly used MAD scaling factor.

[0148] S402. Filter and smooth the criticality level to obtain the smoothed criticality level.

[0149] In the practical application of this invention, since data noise in a single frame may cause fluctuations in the estimation of criticality, this invention employs, but is not limited to, low-pass filters and other means to perform smoothing processing on the criticality sequence limited to the same round, in order to suppress single-frame noise.

[0150] S403. Based on a preset threshold, the criticality level after smoothing is divided into critical moments and non-critical moments.

[0151] For example, a binary classification threshold θ can be determined using a predefined threshold or through a self-supervised method (such as the Otsu method). The criticality after smoothing... If the time of the anchor point is greater than or equal to the threshold θ, it is marked as a critical moment; otherwise, it is marked as a non-critical moment.

[0152] S404. Merge consecutive key moments within the same round into candidate intervals.

[0153] Specifically, within the same round of demonstration trajectory, moments that are consecutively marked as critical moments are merged to form several candidate intervals. Each candidate interval contains a continuous segment of critical moments.

[0154] S405. Filter the candidate intervals to obtain the key action intervals.

[0155] The filtering process includes deleting candidate intervals whose duration is less than a preset duration threshold, and merging adjacent candidate intervals whose interval is less than a preset interval threshold to obtain the key action interval.

[0156] Specifically, candidate intervals with durations shorter than a preset duration threshold are deleted to avoid false key detection results caused by brief fluctuations. Adjacent candidate intervals with intervals shorter than a preset interval threshold are merged to avoid incorrectly dividing actual continuous key action intervals into two independent intervals due to minor non-critical time gaps. The remaining candidate intervals after the above screening process are the finally determined key action intervals.

[0157] Optionally, in another embodiment of the present invention, after step S106, one implementation of the robot key motion detection method further includes:

[0158] Training weights for anchor moments are generated based on their criticality.

[0159] Among them, the training weights are positively correlated with the degree of criticality.

[0160] In this embodiment of the invention, the degree of criticality can be mapped to a positive weight, and the mapping method can be as follows:

[0161] ;in, It is a predefined criticality coefficient. It is a predefined temperature coefficient, and together the two determine the magnitude of the impact of the criticality on the weight.

[0162] Optionally, in another embodiment of the invention, the weights can be normalized to a mean of 1 in each task or round to avoid equivalent changes in the learning rate due to changes in the overall loss scale alone.

[0163] When a policy action block covers multiple anchor points, the training weights of that action block can be formed based on its criticality within the coverage area.

[0164] Optionally, in another embodiment of the present invention, one implementation of the robot key motion detection method further includes:

[0165] Save the demonstration round identifier, frame index, criticality level, critical action interval identifier, and training weights corresponding to each anchor point as independent metadata.

[0166] In this embodiment of the invention, after the action domain width estimator completes the scoring, it saves the demonstration round identifier, frame index, criticality level, key action interval identifier, and training weights corresponding to each anchor point as independent metadata. This metadata can be stored in a structured file format (such as JSON or CSV) on a local storage medium or a network storage system. This metadata is used for subsequent robot imitation learning policy training and does not require the policy model to have the same network structure as the domain width estimator.

[0167] Optionally, in another embodiment of the present invention, one implementation of the robot key motion detection method further includes:

[0168] Obtain independent metadata; train the target robot policy model based on the training weights in the independent metadata.

[0169] Specifically, the system reads the saved metadata file from the storage medium to obtain independent metadata, and then loads it into the training system's memory for later use.

[0170] The network parameters of the target robot strategy model are independent of the network parameters of the width estimator used to determine the width of the local actionable domain.

[0171] In this embodiment of the invention, the target robot strategy model A different network structure and parameters are used compared to the width estimator. During training, the following weighted loss function can be used. :

[0172] ;in, The weights are obtained from the previous step based on their criticality. It is the action prediction loss function of the target robot strategy model.

[0173] It should be noted that during the training process, it can also be done according to... Change the frequency of sample selection. The parameters of the action domain width estimator are not jointly updated with the policy loss to avoid the current prediction error of the policy altering the definition of task criticality. After policy training is complete, only deploy... This design ensures that the evaluation results of the critical action interval are not affected by the specific implementation of the strategy model, making it an independent data base that can be used by target strategy models with different structures.

[0174] This invention performs offline key action detection on a demonstration dataset in robot imitation learning and skill transfer scenarios, generating reusable weight metadata to guide subsequent policy model training by focusing on key aspects of the task. It is particularly suitable for complex operation datasets containing multiple scenarios and tasks, aiming to fundamentally improve training efficiency and final operational performance, rather than relying on indirect estimations from human experience or policy models.

[0175] As can be seen from the above scheme, this invention provides a method for detecting key robot actions. By introducing a posterior task result descriptor as a constraint condition during the training of the conditional action distribution model, the model no longer predicts actions solely based on historical conditions. Instead, it learns to determine the conditional action distribution corresponding to the action sequence given historical conditions and the posterior task result descriptor. Based on this, it uses local action modalities in the conditional action distribution to distinguish different action schemes that can be substituted for each other under the same task result. Furthermore, it determines the width of the feasible action domain of the local region to which the current demonstration action belongs based on the local action modality. This width directly represents the range of allowed variation of the action while maintaining the same task result; the narrower the width, the stricter the task's precision constraint on the action. Therefore, key actions can be automatically detected from demonstration data without relying on any pre-trained strategy model or manual rules. The detected key actions directly address the core aspects of task success or failure, effectively filtering out non-critical signals such as pauses and jitter.

[0176] Another embodiment of the present invention provides a robot key motion detection device, such as... Figure 5 As shown, it specifically includes:

[0177] The trajectory acquisition unit 501 is used to acquire the robot's demonstration trajectory.

[0178] The demonstration trajectory includes state and action information at multiple points in time.

[0179] Data construction unit 502 is used to construct historical conditions, posterior task result descriptors, and action sequences after the anchor point time.

[0180] Among them, the historical conditions are constructed based on the state and action information before the anchor point time, and the posterior task result description is constructed based on the state information after the anchor point time.

[0181] The distribution training unit 503 is used to train the conditional action distribution model using sample groups constructed with historical conditions, posterior task result descriptors, and action sequences, to obtain the trained conditional action distribution model.

[0182] The trained conditional action distribution model determines the conditional action distribution corresponding to the action sequence based on historical conditions and posterior task result descriptions; the conditional action distribution contains multiple local action modalities.

[0183] The modality determination unit 504 is used to determine the local action modality to which the current demonstration action sequence belongs based on the conditional action distribution.

[0184] The width determination unit 505 is used to determine the width of the local action domain of the local region where the current demonstration action sequence is located, based on the local action modality to which the current demonstration action sequence belongs.

[0185] The critical action interval determination unit 506 is used to determine the criticality of the anchor point moment based on the width of the local movable action domain, and to determine the critical action interval according to the criticality.

[0186] For details on the specific operation of the units disclosed in the above embodiments of the present invention, please refer to the corresponding method embodiments, such as... Figure 1 As shown, it will not be elaborated further here.

[0187] Optionally, in another embodiment of the present invention, one implementation of the distributed training unit 503 includes:

[0188] The sample group composition unit is used to form a sample group by combining historical conditions, posterior task result descriptors, and action sequences.

[0189] The occlusion processing unit is used to perform occlusion processing on the target action sequence and the action data in the adjacent isolation area of ​​the target action sequence in the sample group using the bidirectional occlusion Transformer model, so as to obtain the occluded data.

[0190] The Conditional Action Distribution Model Training Unit is used to train the Conditional Action Distribution Model based on the occlusion-processed data, and obtain the trained Conditional Action Distribution Model.

[0191] The specific working process of the units disclosed in the above embodiments of the present invention can be found in the corresponding method embodiments, and will not be repeated here.

[0192] Optionally, in another embodiment of the present invention, one implementation of the modal determination unit 504 includes:

[0193] The current demonstration action sequence acquisition unit is used to acquire the current demonstration action sequence.

[0194] The posterior responsibility calculation unit is used to calculate the posterior responsibility of the current demonstration action sequence relative to each local action modality in the conditional action distribution.

[0195] The selection unit is used to select the local action modality corresponding to the largest posterior responsibility as the local action modality to which the current demonstration action sequence belongs.

[0196] The specific working process of the units disclosed in the above embodiments of the present invention can be found in the corresponding method embodiments, and will not be repeated here.

[0197] Optionally, in another embodiment of the present invention, one implementation of the width determining unit 505 includes:

[0198] The direction-related covariance matrix acquisition unit is used to obtain the direction-related covariance matrix within the local action modality to which the current demonstration action sequence belongs.

[0199] The local movable domain width determination unit is used to determine the local movable domain width of a local region based on the determinant of the direction-related covariance matrix.

[0200] Among them, the smaller the determinant value of the direction-dependent covariance matrix, the narrower the corresponding local action domain width.

[0201] The specific working process of the units disclosed in the above embodiments of the present invention can be found in the corresponding method embodiments, and will not be repeated here.

[0202] Optionally, in another embodiment of the present invention, one implementation of the key action range determination unit 506 includes:

[0203] The standardization processing unit is used to standardize the width of the local actionable domain to obtain the criticality of the anchor point moment.

[0204] Among them, the degree of criticality is negatively correlated with the width of the local actionable domain.

[0205] The filtering and smoothing unit is used to filter and smooth the criticality level to obtain the smoothed criticality level.

[0206] The segmentation unit is used to divide the criticality after smoothing into critical moments and non-critical moments based on a preset threshold.

[0207] The merging unit is used to merge consecutive key moments within the same round into candidate intervals.

[0208] The filtering unit is used to filter candidate intervals to obtain key action intervals.

[0209] The filtering process includes deleting candidate intervals whose duration is less than a preset duration threshold, and merging adjacent candidate intervals whose interval is less than a preset interval threshold to obtain the key action interval.

[0210] The specific working process of the units disclosed in the above embodiments of the present invention can be found in the corresponding method embodiments, and will not be repeated here.

[0211] Optionally, in another embodiment of the present invention, one implementation of the robot key motion detection device further includes:

[0212] The training weight generation unit is used to generate training weights for anchor moments based on their criticality.

[0213] Among them, the training weights are positively correlated with the degree of criticality.

[0214] The specific working process of the units disclosed in the above embodiments of the present invention can be found in the corresponding method embodiments, and will not be repeated here.

[0215] Optionally, in another embodiment of the present invention, one implementation of the robot key motion detection device further includes:

[0216] The storage unit is used to save the demonstration round identifier, frame index, criticality, critical action interval identifier, and training weights corresponding to each anchor point as independent metadata.

[0217] The specific working process of the units disclosed in the above embodiments of the present invention can be found in the corresponding method embodiments, and will not be repeated here.

[0218] Optionally, in another embodiment of the present invention, one implementation of the robot key motion detection device further includes:

[0219] Independent metadata retrieval unit, used to retrieve independent metadata.

[0220] The robot policy model training unit is used to train the target robot policy model based on the training weights in the independent metadata.

[0221] The network parameters of the target robot strategy model are independent of the network parameters of the width estimator used to determine the width of the local actionable domain.

[0222] The specific working process of the units disclosed in the above embodiments of the present invention can be found in the corresponding method embodiments, and will not be repeated here.

[0223] As can be seen from the above scheme, the present invention provides a robot key motion detection device. By introducing a posterior task result descriptor as a constraint condition during the training of the conditional motion distribution model, the model no longer predicts actions solely based on historical conditions. Instead, it learns to determine the conditional motion distribution corresponding to the action sequence given historical conditions and the posterior task result descriptor. Based on this, it uses local motion modalities in the conditional motion distribution to distinguish different action schemes that can be substituted for each other under the same task result. Furthermore, it determines the width of the feasible action domain of the local region to which the current demonstration action belongs based on the local motion modality. This width directly represents the range of allowed variation of the action while maintaining the same task result; the narrower the width, the stricter the task's precision constraint on the action. Therefore, key actions can be automatically detected from demonstration data without relying on any pre-trained strategy model or manual rules. The detected key actions directly address the core aspects of task success or failure, effectively filtering out non-critical signals such as pauses and jitter.

[0224] Another embodiment of the present invention provides an electronic device, such as... Figure 6 As shown, it includes:

[0225] One or more processors 601.

[0226] Storage device 602, on which one or more programs are stored.

[0227] When the one or more programs are executed by the one or more processors 601, the one or more processors 601 implement the robot key motion detection method as described in the above embodiments.

[0228] Another embodiment of the present invention provides a computer storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the robot key motion detection method as described in the above embodiments.

[0229] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0230] It should be noted that the computer-readable medium described above in this invention can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0231] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0232] Another embodiment of the present invention provides a computer program product, which, when executed, is used to perform the above-described robot key motion detection method.

[0233] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, it performs the functions defined in the methods of the embodiments of the present invention.

[0234] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in this invention is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely exemplary forms for implementing the invention.

[0235] While several specific implementation details are included in the foregoing discussion, these should not be construed as limiting the scope of the invention. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0236] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention is not limited to the specific combination of the above-described technical features, but also includes other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above-described features with technical features of the present invention (but not limited to) that have similar functions.

Claims

1. A method for detecting key actions of a robot, characterized in that, include: Obtain the robot's demonstration trajectory; wherein the demonstration trajectory includes state information and action information at multiple moments; For each anchor point moment, construct historical conditions, posterior task result descriptors, and action sequences following the anchor point moment; wherein, the historical conditions are constructed based on state information and action information before the anchor point moment, and the posterior task result descriptors are constructed based on state information following the anchor point moment. A conditional action distribution model is trained using a sample group constructed with the historical conditions, the posterior task result descriptor, and the action sequence to obtain a trained conditional action distribution model; wherein, the trained conditional action distribution model determines the conditional action distribution corresponding to the action sequence based on the historical conditions and the posterior task result descriptor; the conditional action distribution includes multiple local action modalities; Based on the conditional action distribution, determine the local action modality to which the current demonstration action sequence belongs; Based on the local action modality to which the current demonstration action sequence belongs, determine the width of the local action domain of the local region where the current demonstration action sequence is located; The criticality of the anchor point moment is determined based on the width of the local action domain, and the critical action range is determined based on the criticality.

2. The robot key motion detection method according to claim 1, characterized in that, The conditional action distribution model is trained using the sample group constructed with the historical conditions, the posterior task result descriptor, and the action sequence to obtain the trained conditional action distribution model, including: The historical conditions, the posterior task result descriptor, and the action sequence constitute a sample group; The bidirectional occlusion Transformer model is used to occlude the target action sequence in the sample group and the action data in the adjacent isolation area of ​​the target action sequence to obtain the occluded data. The conditional action distribution model is trained based on the occlusion-processed data to obtain the trained conditional action distribution model.

3. The robot key motion detection method according to claim 1, characterized in that, Determining the local action modality to which the current demonstration action sequence belongs based on the conditional action distribution includes: Obtain the current demonstration action sequence; Calculate the posterior responsibility of the current demonstration action sequence relative to each local action modality in the conditional action distribution; The local action modality corresponding to the largest posterior responsibility is selected as the local action modality to which the current demonstration action sequence belongs.

4. The robot key motion detection method according to claim 1, characterized in that, The step of determining the width of the local action domain of the local region where the current demonstration action sequence is located, based on the local action modality to which the current demonstration action sequence belongs, includes: Obtain the direction-related covariance matrix within the local action modality to which the current demonstration action sequence belongs; The width of the local movable domain of the local region is determined based on the determinant of the direction-related covariance matrix; wherein, the smaller the determinant value of the direction-related covariance matrix, the narrower the corresponding local movable domain width.

5. The robot key motion detection method according to claim 1, characterized in that, The process of determining the criticality of the anchor point moment based on the width of the local movable action domain, and determining the critical action interval based on the criticality, includes: The width of the local movable action domain is standardized to obtain the criticality of the anchor point moment; wherein the criticality is negatively correlated with the width of the local movable action domain. The criticality level is then filtered and smoothed to obtain a smoothed criticality level. The criticality of the smoothed process is divided into critical moments and non-critical moments based on a preset threshold. Merge consecutive key moments within the same round into candidate intervals; The candidate intervals are filtered to obtain key action intervals; wherein, the filtering process includes deleting candidate intervals whose duration is less than a preset duration threshold, and merging adjacent candidate intervals whose interval is less than a preset interval threshold to obtain the key action intervals.

6. The robot key motion detection method according to claim 1, characterized in that, After determining the criticality of the anchor point moment based on the width of the local movable action domain, the method further includes: Training weights for the anchor point are generated based on the criticality level; wherein the training weights are positively correlated with the criticality level.

7. The robot key motion detection method according to claim 6, characterized in that, After generating the training weights for the anchor point time based on the criticality, the method further includes: Save the demonstration round identifier, frame index, criticality level, critical action interval identifier, and training weights corresponding to each anchor point as independent metadata.

8. The robot key motion detection method according to claim 7, characterized in that, After saving the demonstration round identifier, frame index, criticality level, key action interval identifier, and training weights corresponding to each anchor point time as independent metadata, the method further includes: Obtain the independent metadata; The target robot policy model is trained based on the training weights in the independent metadata; wherein the network parameters of the target robot policy model are independent of the network parameters of the width estimator used to determine the width of the local action domain.

9. A robot key motion detection device, characterized in that, include: A trajectory acquisition unit is used to acquire the robot's demonstration trajectory; wherein, the demonstration trajectory includes state information and action information at multiple moments; A data construction unit is used to construct historical conditions, posterior task result descriptors, and action sequences after the anchor point time; wherein, the historical conditions are constructed based on state information and action information before the anchor point time, and the posterior task result descriptors are constructed based on state information after the anchor point time. A distribution training unit is used to train a conditional action distribution model using a sample group constructed with the historical conditions, the posterior task result descriptor, and the action sequence, to obtain a trained conditional action distribution model; wherein, the trained conditional action distribution model determines the conditional action distribution corresponding to the action sequence based on the historical conditions and the posterior task result descriptor; the conditional action distribution includes multiple local action modalities. The modality determination unit is used to determine the local action modality to which the current demonstration action sequence belongs based on the conditional action distribution. The width determination unit is used to determine the width of the local action domain of the local region where the current demonstration action sequence is located, based on the local action modality to which the current demonstration action sequence belongs. The critical action interval determination unit is used to determine the criticality of the anchor point moment based on the width of the local movable action domain, and to determine the critical action interval according to the criticality.

10. A computer storage medium, characterized in that, It stores a computer program, wherein the computer program, when executed by a processor, implements the robot key motion detection method as described in any one of claims 1 to 8.

Images