Shadow-reasoning-based industrial embodied robot teleoperation data end-side self-screening method

Abstract

The application discloses a kind of industrial body robot remote control data end side self-screening methods based on shadow reasoning.There is:1, with visual frame timestamp as benchmark, construct " visual frame-action instruction " cross-modal data tuple stream, and continuous action vector is discrete mapping as real action Token vector;2, construct shadow reasoning model, generate prediction probability distribution matrix based on cross-modal data tuple;3, according to real action Token vector and prediction probability distribution matrix, calculate prediction deviation factor and endogenous uncertainty factor, and frame-level value degree is fused, and frame-level value degree is continuously satisfied preset condition Data tuples are packaged as initial screening sample segments;4, calculate the feature difference between initial screening sample segments and retained segments, and determine the retention or discard of initial screening sample segments by double threshold decision.The application can realize low overhead, efficient remote control data screening on the end side, and improve the data quality of body intelligent model training.

Description

Technical Field

[0001] This invention belongs to the field of industrial embodied robot technology, and mainly relates to a self-selection method for remote control data of industrial embodied robots based on shadow reasoning. Background Technology

[0002] In recent years, with the rapid development of artificial intelligence and robotics, embodied robotics technology has evolved from early laboratory research to industrial application trials and partial scenario implementation. Many institutions are actively developing and testing embodied intelligence technologies, including cross-modal perception, decision-making, and intelligent computing, to achieve autonomous decision-making and real-time environmental interaction. Among these, the acquisition and screening of high-quality remote control data is a core element of embodied intelligence model training. Robots need to learn complex action logic through massive amounts of teaching data to safely and effectively perform tasks in industrial environments.

[0003] However, current industrial embodied robots still suffer from the following shortcomings in data acquisition and screening technologies: traditional full-data acquisition methods generate significant data redundancy during prolonged remote operation, leading to a severe waste of storage and bandwidth resources; in complex industrial operation scenarios, fluctuations in the operator's actions or environmental interference may result in inconsistent quality of some acquired data, making it difficult for existing systems to evaluate the training value of the data in real time at the edge; when relying on embodied models for prediction, the limited generalization ability of the model in the face of unseen scenarios means that the inability to capture and screen out difficult examples with model prediction failures or low confidence levels will directly affect the efficiency of subsequent model iteration and optimization. Therefore, how to utilize lightweight models to perform real-time quality assessment and uncertainty analysis of cross-modal data during remote operation under limited edge computing power, and accurately retain difficult example segments with high feature differences to reduce redundancy in the data construction process and improve model training quality, has become an urgent problem to be solved. Summary of the Invention

[0004] To overcome the shortcomings of the prior art, this invention proposes a self-selection method for remote control data of industrial embodied robots based on shadow reasoning. This method aims to improve the quality of remote control data of industrial embodied robots, enhance the ability of industrial embodied robots to mine difficult data, and reduce the redundancy of end-side storage and bandwidth resources, thereby providing a new approach for data engineering of embodied intelligence.

[0005] To solve the above problems, the present invention adopts the following technical solution: Step 1: Collect visual observations from the robot using the visual frame timestamp as a unified time reference. Instruction embedding and standardized motion vectors Construct a unified cross-modal data tuple stream that includes vision, commands, and actions. ; Step 2: Construct a mapping rule from action vectors to discrete action tokens, which will convert continuous standardized action vectors... Mapped to the corresponding real action token vector Generate a sequence of real action tokens in chronological order. }; Step 3: Construct a shadow reasoning model based on the historical context window. As input, the current time step is output after two-layer matrix mapping processing. The predicted probability distribution matrix below ; Step 4: Construct the evaluation model and input the time steps. Corresponding real action token vector With the predicted probability distribution matrix Calculate the prediction bias factor With endogenous uncertainty factors .

[0006] Step 5: Utilize the prediction bias factor With endogenous uncertainty factors Building frame-level value and in the cross-modal data tuple stream The above combines a continuous hit mechanism to encapsulate the fragments to be retained. Calculate the fragment Feature difference factor relative to the existing set of fragments The decision to include the fragment is made using a dual-threshold method. Its representation vector is written to the end-side storage space.

[0007] Compared with existing technologies, the beneficial effects of the present invention are as follows: 1. This invention proposes a self-selection method for remote control data of industrial embodied robots based on shadow reasoning. This method evaluates the value of cross-modal data in the remote control process in real time and accurately. Based on the support degree and distribution uncertainty of real action tokens in the prediction probability distribution, this method quantifies the value of data and can efficiently mine difficult example data with failed or low confidence predictions of shadow reasoning models. This eliminates a large number of simple and repetitive redundant data sources for the training of subsequent embodied intelligent models and lays a high-quality data foundation.

[0008] 2. This invention designs a data value quantification calculation method based on probability distribution analysis. It selects the prediction probability distribution matrix as the core judgment factor, and calculates the prediction deviation factor and endogenous uncertainty factor by analyzing the likelihood and information entropy of the predicted action token and the real action token, respectively, and finally forms a frame-level value, which accurately reflects the capability boundary and confidence distribution of the model in the current scenario, thereby providing a reliable quantitative basis for the triggering of the fragments to be retained in the data screening process.

[0009] 3. This invention provides a dynamic data retention strategy for industrial robots in environments with limited end-side storage resources. When a large amount of remote control data is generated and the device's storage space is limited, the feature difference factor between the fragment to be retained and the already retained set can be calculated. A dual threshold decision based on value and feature difference can be performed, and a circular buffer mechanism can be used to dynamically encapsulate high-value fragments. This can effectively reduce the redundancy ratio in the data construction process and ensure the diversity of samples, thereby realizing efficient self-screening and storage of end-side data and opening up a new optimization strategy for the development of data engineering for embodied intelligence. Attached Figure Description

[0010] Figure 1 This is a basic flowchart of the invention based on edge data segment triggering and dual threshold retention decision; Figure 2 This is an overall flowchart of the self-selection method for remote control data on the edge of an industrial embodied robot based on shadow reasoning, according to the present invention. Figure 3 This is a joint distribution and comprehensive value mapping diagram of frame-level prediction bias factor and uncertainty factor in an embodiment of the present invention. Detailed Implementation

[0011] In this embodiment, a self-selection method for remote control data of an industrial embodied robot based on shadow reasoning is described, such as... Figure 1 and Figure 2 As shown, the procedure is as follows: Step 1: Collect visual observations from the robot using the visual frame timestamp as a unified time reference. Instruction embedding and standardized motion vectors Construct a unified cross-modal data tuple stream that includes vision, commands, and actions. ; Step 2: Construct a mapping rule from action vectors to discrete action tokens, which will convert continuous standardized action vectors... Mapped to the corresponding real action token vector Generate a sequence of real action tokens in chronological order. }; Step 3: Construct a shadow reasoning model based on the historical context window. As input, the current time step is output after two-layer matrix mapping processing. The predicted probability distribution matrix below ; Step 4: Construct the evaluation model and input the time steps. Corresponding real action token vector With the predicted probability distribution matrix Calculate the prediction bias factor With endogenous uncertainty factors .

[0012] Step 5: Utilize the prediction bias factor With endogenous uncertainty factors Building frame-level value and in the cross-modal data tuple stream The above combines a continuous hit mechanism to encapsulate the fragments to be retained. Calculate the fragment Feature difference factor relative to the existing set of fragments The decision to include the fragment is made using a dual-threshold method. Its representation vector is written to the end-side storage space.

[0013] The following is a detailed explanation: Step 1: Collect visual observations from the robot using the visual frame timestamp as a unified time reference. Instruction embedding and standardized motion vectors Construct a unified cross-modal data tuple stream that includes vision, commands, and actions. ; Step 1.1: Using the visual frame timestamps output by the visual sensor as a unified time reference, let the visual frame frequency be... Then the visual frame index The corresponding time step is ,in , This refers to the time steps in remote data acquisition; in this embodiment, the industrial robot is a six-degree-of-freedom industrial robotic arm, the end effector is an electric gripper, and the onboard vision sensor is a fixedly mounted RGB camera. Visual frame rate. Visual frame index .

[0014] Step 1.2: Obtain the high-frequency raw absolute pose state flow of the robot's end effector. ,in, Represents the end position vector; Describe the quaternion of the end attitude and satisfy ; Indicates the state of the end gripper; This is the timestamp for the controller sampling.

[0015] Step 1.2.1, in the original state flow Searching in the middle satisfies For the sample pairs, calculate each time step using equation (1). The increment.

[0016] (1) In equation (1), These are the normalized time interpolation coefficients. , , These are the end-effector position increment vector, the end-effector attitude quaternion increment, and the gripper state increment, respectively. , ， ; This is quaternion multiplication; Represents the spherical linear interpolation of unit quaternions; Convert to Euler angle increments to obtain the rotation increment vector. In this embodiment, the position increment, rotation increment, and gripper increment corresponding to the first frame are preset to zero vectors; the end position increment... With attitude increment All are expressed in the robot base coordinate system; the quaternion increments When converting to Euler angle increments, the preset ZYX rotation sequence is used.

[0017] Step 1.2.2: Define the original action vector. A truncation mapping function is used for normalization to generate standardized action vectors. The calculation formula is as follows: (2) In equation (2), Indicates time step Standardized motion vectors The Dimensional components; This represents the truncation function; Represents the original action vector The 1D physical quantity components; and They represent the first Minimum and maximum truncation thresholds for dimensional physical quantities; This represents the total number of dimensions in the action vector. The components are combined in dimensional order to obtain the standardized action vector. ,in, In this embodiment, the standardized motion vector has 7 dimensions, including a three-dimensional end-effector position increment, a three-dimensional end-effector attitude Euler angle increment, and a one-dimensional gripper opening / closing increment. The position increment truncation threshold is 0.02m, and the Euler angle increment truncation threshold is 0.06rad.

[0018] Step 1.3: Construct a visual frame sequence under a unified time reference. and instruction stream ; Step 1.3.1: Obtain the visual frame sequence output by the airborne vision sensor under a unified time reference. ,in, In time step The visual observations obtained.

[0019] Step 1.3.2: Obtain the instruction stream Take the current time The instruction from the most recent update is used as the current embedding. ,in Update the timestamp for the instruction. The corresponding instruction is vectorized and embedded.

[0020] Step 1.4: Visual observations under a unified time reference Instruction embedding With standardized motion vectors Index by Time Perform tuple associations to construct a unified cross-modal data tuple stream: .

[0021] Step 2: Construct a mapping rule from action vectors to discrete action tokens, which will convert continuous standardized action vectors... Mapped to the corresponding real action token vector Generate a sequence of real action tokens in chronological order. }; Step 2.1: Given the dimension of the action vector Number of discrete symbols per dimension Define the discrete action domain:

[0022] in, Represents a discrete action token vector; Indicates the first Discrete Token indexes for each action dimension; This represents the discrete granularity of each token dimension.

[0023] Step 2.2, regarding the first Each action dimension defines the quantized binning boundary sequence. ;in, , , The boundary sequence Used to determine the mapping relationship between the physical numerical range of this dimension and the discrete token index; For the first The discrete token index after quantization and binning for each action dimension. In this embodiment, the discrete granularity of each action dimension is uniformly set to 256 discrete tokens, and the quantization binning boundary sequence is constructed using an equal-width binning method. For any time step, the first... Standardized motion components, when their values ​​fall into the 1st dimension When there are multiple bin intervals, the corresponding real discrete action Token index is denoted as: .

[0024] Step 2.3, regarding the time step The Dimensional action components In the bin boundary sequence In the interval search, the corresponding real discrete token index is determined according to equation (3). And combine the indexes of each dimension to obtain the time step. Real Action Token Vector Construct a sequence of real action tokens in chronological order. }, (3).

[0025] Step 3: Construct a shadow reasoning model based on the historical context window. As input, the current time step is output after two-layer matrix mapping processing. The predicted probability distribution matrix below ; Step 3.1: For each time step under a unified time base Read the current observations and instructions from the cross-modal tuple stream to obtain The construction length is History Context Window The first one in the window Time series elements ( Construct according to formula (4): (4) In equation (4), The default padding item is configured as a zero-padding item. or first item In this embodiment, the length of the history context window is set. .

[0026] Step 3.2, Deployment parameters are: Lightweight shadow inference model Output space and Maintain consistency. For any valid time interval. ,definition Define window stitching and fusion vector ,in Represents the dimensions of visual features. This indicates the dimension of instruction embedding. For timing splicing operators, For visual representation extraction network, the predicted probability distribution matrix is ​​output using equation (5): (5) In equation (5), This represents the normalized exponential mapping function for each action dimension. This is the preset activation function; And cross-modal fusion matrix Hidden layer bias vector Action distribution projection matrix Output bias vector , Indicates the dimension of hidden layer features; Indicates rearrangement as In this embodiment, the visual representation extraction network uses a lightweight convolutional neural network with a hidden layer dimension of 512. The input image is compressed into a 256-dimensional visual feature vector through convolutional feature extraction, global average pooling, and fully connected layers. The instruction embedding is obtained by mapping a preset task template through an embedding lookup table with a dimension of 128. The activation function used is the ReLU function.

[0027] Step 3.3: Predict the probability distribution matrix Used to characterize the confidence distribution of the model for discrete token values ​​of each action dimension in the current state: (6) In equation (6), Indicates at time step Under the conditions The value of the dimension action token is The predicted probabilities satisfy the row normalization constraint: .

[0028] Step 4: Construct the evaluation model and input the time steps. Corresponding real action token vector With the predicted probability distribution matrix Calculate the prediction bias factor With endogenous uncertainty factors .

[0029] Step 4.1: Input time step The real action token vector and the one-dimensional probability distribution matrix output by the shadow inference model ; Step 4.2: Evaluate the model by calling the prediction bias evaluation operator and constructing the prediction bias factor (7). (7) In equation (7), Indicates time step Next Authentic Action Token Value Retrieval Indicates at time step Conditions, No. The value of the action token is the predicted probability of the token corresponding to the actual action. The preset minimum positive number; in this embodiment, .

[0030] Step 4.3: Evaluate the model by calling the endogenous uncertainty evaluation operator and constructing the endogenous uncertainty factor of formula (8). (8) In equation (8), Indicates the first The value of the dimension action token is The predicted probability.

[0031] Step 5: Construct frame-level value using the prediction bias factor and the endogenous uncertainty factor. and in the cross-modal data tuple stream The above combines a continuous hit mechanism to encapsulate the fragments to be retained. Calculate the feature difference factor of the fragment relative to the set of retained fragments. The decision to include the fragment is made using a dual-threshold method. Its representation vector is written to the end-side storage space.

[0032] Step 5.1: Based on the prediction bias factor With endogenous uncertainty factors Build frame-level value (9) In equation (9), To predict the weights of the bias factor, Weights for endogenous uncertainty factors. , , In this example, weighting coefficients are set. , Select 500 frame-level samples from each time step and plot them. and Scattered distribution and frame-level comprehensive value Perform color mapping, such as Figure 3 As shown.

[0033] Step 5.2: Perform continuous hit detection based on frame-level value, encapsulate the data segments to be retained, and define the segment value. Step 5.2.1: Set the frame-level scoring threshold With consecutive hit length parameter If at the current moment Satisfy continuous The frame-level value of the frame exceeds the threshold And the moment before The above continuous requirement was not met. Frame conditions are then used in the tuple stream using equation (10). Marked encapsulated fragment .

[0034] (10) In equation (10), , and To fix the length of the preceding and following text; in this example, the hit length parameter is set. , , , .

[0035] Step 5.2.2, Define the fragment The fragment value score is the maximum value of the frame-level score within the fragment: .

[0036] Step 5.3: Deploy a lightweight observation and characterization extraction network with fixed parameters on the edge computing device. Visual observation of each frame within the segment Extracting low-dimensional representation vectors , for The feature dimensions are used to analyze the fragment using equation (11). Generate fragment representation vectors, the The visual features of the backbone network parameters and step 3.2 shared.

[0037] (11) Step 5.4: Maintain the set of retained fragments on the edge computing device. and its corresponding segment representation vector Treating the remaining fragments Calculate its minimum cosine difference relative to the existing set, and construct the feature difference factor of the fragment using equation (12): (12) In equation (12), For a preset minimum positive number, when At that time, the edge computing device will Set to the default maximum value.

[0038] Step 5.5: Set the feature difference threshold on the edge computing device. With value threshold Using equation (13) for each triggered fragment Execute the binary retention decision function: (13) In equation (13), This is an indicator function that returns 1 if the condition within the parentheses is true, and 0 otherwise. At that time, the edge computing device generates structured data objects. Write to the retained collection ,when Discard End-side computing devices will Configured with a capacity of A circular retention buffer; when Furthermore, when a new segment needs to be written, the edge computing device overwrites the oldest written segment in the writing order, maintaining... In this embodiment, , , .

[0039] In summary, this invention discloses an edge-side self-screening method for remote control data of industrial embodied robots based on shadow reasoning. It includes: 1. Constructing a cross-modal data tuple stream of "visual frame-command-action" based on visual frame timestamps, and discretizing continuous action vectors into real action token vectors; 2. Constructing a shadow reasoning model to generate a prediction probability distribution matrix based on the cross-modal data tuples; 3. Calculating prediction bias factors and endogenous uncertainty factors based on the real action token vectors and the prediction probability distribution matrix, fusing them to obtain frame-level value, and encapsulating data tuples whose frame-level value continuously meets preset conditions as initial screening sample fragments; 4. Calculating the feature differences between the initial screening sample fragments and the retained fragments, and determining whether to retain or discard the initial screening sample fragments through a dual-threshold decision. This invention can achieve low-overhead, high-efficiency remote control data screening at the edge, improving the data quality for training embodied intelligent models.

[0040] In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described above.

[0041] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the shadow reasoning-based remote data terminal self-selection methods for industrial embodied robots described in the above embodiments.

[0042] It is understood that the systems, devices, and storage media provided in the embodiments of the present invention correspond to the methods provided in the embodiments of the present invention, and the explanations, examples, and beneficial effects of the relevant content can be referred to the corresponding parts of the above methods.

[0043] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0044] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0045] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0046] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A self-selection method for remote control data of industrial embodied robots based on shadow reasoning, characterized in that, Includes the following steps: Step 1: Collect visual observations from the robot using the visual frame timestamp as a unified time reference. Instruction embedding and standardized motion vectors Construct a unified cross-modal data tuple stream that includes vision, commands, and actions. ; Step 2: Construct a mapping rule from action vectors to discrete action tokens, which will convert continuous standardized action vectors... Mapped to the corresponding real action token vector Generate a sequence of real action tokens in chronological order. }; Step 3: Construct a shadow reasoning model based on the historical context window. As input, the current time step is output after two-layer matrix mapping processing. The predicted probability distribution matrix below ; Step 4: Construct the evaluation model and input the time steps. Corresponding real action token vector With the predicted probability distribution matrix Calculate the prediction bias factor With endogenous uncertainty factors ; Step 4 includes the following steps: Step 4.1: Input time step The real action token vector and the one-dimensional probability distribution matrix output by the shadow inference model ; Step 4.2: Evaluate the model by calling the prediction bias evaluation operator and constructing the prediction bias factor of formula (7). （7） In equation (7), Indicates time step Next Authentic Action Token Value Retrieval Indicates at time step Conditions, No. The value of the action token is the predicted probability of the token corresponding to the actual action. The minimum positive number is preset. Step 4.3: Evaluate the model by calling the endogenous uncertainty evaluation operator and constructing the endogenous uncertainty factor of formula (8). （8） In equation (8), Indicates the first The value of the dimension action token is The predicted probability; Step 5: Utilize the prediction bias factor With endogenous uncertainty factors Building frame-level value and in the cross-modal data tuple stream The above combines a continuous hit mechanism to encapsulate the fragments to be retained. Calculate the fragment Feature difference factor relative to the existing set of fragments The decision to include the fragment is made using a dual-threshold method. Its representation vector is written to the end-side storage space; Step 5 includes step 5.1, based on the prediction bias factor. With endogenous uncertainty factors Build frame-level value （9） In equation (9), To predict the weights of the bias factor, Weights for endogenous uncertainty factors. , , .

2. The self-selection method for remote control data of industrial embodied robots based on shadow reasoning according to claim 1, characterized in that, Step 1 includes: Step 1.1: Using the visual frame timestamps output by the visual sensor as a unified time reference, let the visual frame frequency be... Then the visual frame index The corresponding time step is ,in , This refers to the number of time steps in remote control data acquisition. Step 1.2: Obtain the high-frequency raw absolute pose state flow of the robot end effector. ,in, Represents the end position vector; Describe the quaternion of the end attitude and satisfy ; Indicates the state of the end gripper; For controller sampling timestamps; Step 1.3: Construct a visual frame sequence under a unified time reference. and instruction stream ; Step 1.3.1: Obtain the visual frame sequence output by the airborne vision sensor under a unified time reference. ,in, In time step The visual observations obtained; Step 1.3.2: Obtain the instruction stream Take the current time step The instruction from the most recent update is embedded as the current instruction. , Update the timestamp for the instruction. The corresponding instruction vectorization embedding representation; Step 1.4: Visual observations under a unified time reference Instruction embedding With standardized motion vectors Index by Time Perform tuple associations to construct a unified cross-modal data tuple stream: .

3. The self-selection method for remote control data of an industrial embodied robot based on shadow reasoning according to claim 2, characterized in that, Step 1.2 includes the following steps: Step 1.2.1, in the original state flow Searching in the middle satisfies For the sample pairs, calculate each time step using equation (1). Increment: (1) In equation (1), These are the normalized time interpolation coefficients. These are the end-effector position increment vector, the end-effector attitude quaternion increment, and the gripper state increment, respectively. , ， ; This is quaternion multiplication; Represents the spherical linear interpolation of unit quaternions; Convert to Euler angle increments to obtain the rotation increment vector. ; Step 1.2.2: Define the original action vector. Normalization is performed using a truncation mapping function to generate standardized action vectors. The calculation formula is as follows: (2) In equation (2), Indicates time step Standardized motion vectors The Dimensional components; This represents the truncation function; Represents the original action vector The 1D physical quantity components; and They represent the first Minimum and maximum truncation thresholds for dimensional physical quantities; The total number of dimensions of the action vector is given by combining the components in dimensional order to obtain the standardized action vector. ,in, .

4. The self-selection method for remote control data of an industrial embodied robot based on shadow reasoning according to claim 3, characterized in that, Step 2 includes the following steps: Step 2.1: Given the total dimension of the action vector Number of discrete symbols per dimension Define the discrete action domain: in, Represents a discrete action token vector; Indicates the first Discrete Token indexes for each action dimension; Step 2.2, regarding the first Each action dimension defines the quantized binning boundary sequence. ;in, , , The boundary sequence Used to determine the mapping relationship between the physical numerical range of this dimension and the discrete token index; For the first Discrete Token Index after binning by action dimension; Step 2.3, regarding the time step The Dimensional action components In the bin boundary sequence In the interval search, the corresponding real discrete token index is determined according to equation (3). And combine the indexes of each dimension to obtain the time step. Real Action Token Vector Construct a sequence of real action tokens in chronological order. }, (3).

5. The self-selection method for remote control data of an industrial embodied robot based on shadow reasoning according to claim 4, characterized in that, Step 3 includes the following steps: Step 3.1: For each time step under a unified time base Read the current observations and instructions from the cross-modal tuple stream to obtain The construction length is History Context Window The first one in the window Time series elements , Construct according to formula (4): (4) In equation (4), The default padding item is configured as a zero-padding item. or first item ; Step 3.2, Deployment parameters are: Lightweight shadow inference model Output space and Maintain consistency; for any valid time. ,definition Define window stitching and fusion vector ,in Represents the dimensions of visual features. This indicates the dimension of instruction embedding. For timing splicing operators, For visual representation extraction networks; Output the predicted probability distribution matrix using equation (5): (5) In equation (5), This represents the normalized exponential mapping function for each action dimension. This is the preset activation function; And cross-modal fusion matrix Hidden layer bias vector Action distribution projection matrix Output bias vector , Indicates the dimension of hidden layer features; Indicates rearrangement as matrix; Step 3.3: Predict the probability distribution matrix Used to characterize the confidence distribution of the model for discrete token values ​​of each action dimension in the current state: (6) In equation (6), Indicates at time step Under the conditions The value of the dimension action token is The predicted probabilities satisfy the row normalization constraint: .

6. The self-selection method for remote control data of an industrial embodied robot based on shadow reasoning according to claim 5, characterized in that, Step 5 also includes the following steps: Step 5.2: Perform continuous hit detection based on frame-level value, encapsulate the data segments to be retained, and define the segment value. Step 5.2.1: Set the frame-level scoring threshold With consecutive hit length parameter If at the current moment Satisfy continuous The frame-level value of the frame exceeds the threshold And the moment before The above continuous requirement was not met. Frame conditions are then used in the tuple stream using equation (10). Marked encapsulated fragment ; （10） In equation (10), , and To fix the length of the preceding and following text; Step 5.2.2, Define the fragment The fragment value score is the maximum value of the frame-level score within the fragment: ; Step 5.3: Deploy a lightweight observation and characterization extraction network with fixed parameters on the edge computing device. Visual observation of each frame within the segment Extracting low-dimensional representation vectors , for The feature dimensions are used to analyze the fragment using equation (11). Generate fragment representation vectors: （11） Step 5.4: Maintain the set of retained fragments on the edge computing device. and its corresponding segment representation vector Treatment of retained fragments Calculate its minimum cosine difference relative to the existing set, and construct the feature difference factor of the fragment using equation (12): （12） In equation (12), For a preset minimum positive number, when At that time, the edge computing device will Set to the preset maximum value; Step 5.5: Set the feature difference threshold on the edge computing device. With value threshold Using equation (13) for each triggered fragment Execute the binary retention decision function: （13） In equation (13), This is an indicator function; it takes the value 1 if the condition within the parentheses is true, and 0 otherwise. At that time, the edge computing device generates structured data objects. Write to the retained collection ,when Discard The edge computing device will collect the fragment set. Configured with a capacity of A circular retention buffer; when Furthermore, when a new segment needs to be written, the edge computing device overwrites the oldest written segment in the writing order, maintaining... .

Images