A diffusion model-based vla action sequence random generation method and system

Abstract

The present application relates to the technical field of artificial intelligence and embodied intelligent robot control, and particularly relates to a VLA action sequence random generation method and system based on a diffusion model, comprising: acquiring a visual image, a language instruction, a joint motion state and a moving obstacle trajectory matrix; extracting features and mapping to generate a semantic guidance vector, while constructing a control obstacle function; sampling noise to start reverse denoising iteration, and extracting a normal vector gradient matrix by taking the partial derivative of the control obstacle function; constructing a null space projection operator based on this, and performing orthogonal mapping on the task score gradient output by the denoising network to extract a safe task gradient; solving a safe compensation scalar to weight the normal vector gradient matrix, and superimposing it on the safe task gradient to cyclically update the action state and output a target action sequence; and finally converting into a driving instruction to control an execution device. The present application decouples the feature coupling of semantic intent and obstacle avoidance gradient through orthogonal mapping, thereby avoiding the device from falling into a space-time deadlock in dynamic interaction.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and embodied intelligent robot control technology, and in particular to a method and system for random generation of VLA action sequences based on a diffusion model. Background Technology

[0002] In dynamic human-machine collaborative work scenarios, the execution equipment needs to complete continuous physical operations based on natural language instructions and visual perception of the environment. Currently, the industry generally adopts a visual language action large model combined with a diffusion strategy to reconstruct and generate action sequences. However, when there are randomly moving obstacles in the work space, the semantic task features that tend towards the work target generated by the traditional diffusion denoising network and the repulsive features that move away from the moving obstacles imposed by the external safety system will directly couple with time-varying vectors in the same high-dimensional action manifold space. As the obstacles continuously generate physical displacement, the superposition of features in the same direction will cause the system to generate opposite gradient adversarial between executing the task and avoiding obstacles during the inverse iterative solution process. This will cause the underlying servo system to fall into a dynamic spatiotemporal deadlock, which will then cause the execution trajectory of the execution equipment to oscillate violently, or even cause direct physical collisions with moving obstacles at the trajectory intersection, resulting in the interruption of assembly line operations.

[0003] Specifically, traditional diffusion control schemes typically employ a serial architecture of 'generation-post-processing'. This involves first generating a complete sequence of actions from the diffusion model, then feeding it into a separate collision detection module or artificial potential field module for safety verification. When a collision risk is detected, it is usually added as an external penalty term to the loss function. While this external verification method achieves basic obstacle avoidance, it results in action generation and safety constraints existing in separate computational manifold spaces. This leads to a situation where the execution device faces adversarial conflicts between task and obstacle avoidance vectors in dynamic environments, resulting in a spatiotemporal deadlock. Summary of the Invention

[0004] To overcome the above shortcomings, this invention provides a method and system for random generation of VLA action sequences based on a diffusion model. It aims to improve the problem that traditional diffusion control mostly uses direct gradient superposition, which causes the execution device to get stuck in a spatiotemporal deadlock in a dynamic environment due to the conflict between the task and obstacle avoidance vectors.

[0005] In a first aspect, the present invention provides the following technical solution: a method for randomly generating VLA action sequences based on a diffusion model, comprising the following steps:

[0006] S1. Acquire visual images, language commands, joint motion states of the target execution device, and trajectory matrix of moving obstacles;

[0007] S2. Input the visual image and the language instruction into the feature extraction network to obtain the original features, use the semantic mapping matrix to map the original features to generate a semantic guidance vector, and at the same time construct a control obstacle function based on the joint motion state and the trajectory matrix of the moving obstacle.

[0008] S3. Sample the initial noise sequence to start the reverse denoising iteration, and in the current iteration step, calculate the partial derivative of the control obstacle function with respect to the current action state, and extract the normal vector gradient matrix. The normal vector gradient matrix indicates the steepest ascent direction pointing to the boundary of the safe area.

[0009] S4. Construct a null space projection operator based on the normal vector gradient matrix, and use the null space projection operator to orthogonally map the task score gradient generated by the denoising network based on the semantic guidance vector to extract the safe task gradient.

[0010] S5. Calculate the safety compensation scalar based on the control obstacle function, superimpose the normal vector gradient matrix weighted by the safety compensation scalar onto the safety task gradient, obtain the update action gradient to update the current action state, repeat the inverse denoising iteration until the stopping condition is met, and output the target action sequence.

[0011] S6. Convert the target action sequence into drive instructions to control the target execution device.

[0012] By adopting the above technical solution, a null space projection operator is constructed to orthogonally map the task score gradient, thereby separating the feature coupling between semantic intent and obstacle avoidance gradient. This improves the problem that traditional diffusion control mostly uses direct gradient superposition, which causes the execution device to get stuck in spatiotemporal deadlock in a dynamic environment due to the adversarial conflict between the task and obstacle avoidance vectors.

[0013] Secondly, this invention provides the following technical solution: a VLA action sequence random generation system based on a diffusion model, comprising the following modules:

[0014] The status data acquisition module is configured to acquire visual images, language commands, joint motion states of the target execution device, and trajectory matrices of moving obstacles.

[0015] The semantic mapping and obstacle construction module is configured to obtain original features by combining the visual image with the language instruction input feature extraction network, generate a semantic guidance vector by mapping the original features using a semantic mapping matrix, and construct a control obstacle function based on the joint motion state and the trajectory matrix of the moving obstacle.

[0016] The noise sampling and gradient extraction module is configured to sample the initial noise sequence to start the inverse denoising iteration, and in the current iteration step, calculate the partial derivative of the control barrier function with respect to the current action state to extract the normal vector gradient matrix, which indicates the steepest ascent direction pointing to the boundary of the safe area;

[0017] The null space orthogonal mapping module is configured to construct a null space projection operator based on the normal vector gradient matrix, and use the null space projection operator to orthogonally map the task score gradient generated by the denoising network based on the semantic guidance vector to extract the safe task gradient.

[0018] The action state reconstruction module is configured to calculate the safety compensation scalar based on the control obstacle function, superimpose the normal vector gradient matrix weighted by the safety compensation scalar onto the safety task gradient, obtain the updated action gradient to update the current action state, loop the inverse denoising iteration until the stopping condition is met, and output the target action sequence.

[0019] The servo drive control module is configured to convert the target action sequence into drive instructions to control the target execution device.

[0020] The present invention has the following beneficial effects:

[0021] 1. In this invention, the task score gradient is orthogonally mapped by constructing a null space projection operator, thereby separating the feature coupling between semantic intent and obstacle avoidance gradient. This improves the problem that traditional diffusion control mostly uses direct gradient superposition, which causes the execution device to get stuck in spatiotemporal deadlock in a dynamic environment due to the adversarial conflict between the task and obstacle avoidance vectors.

[0022] 2. In this invention, by using forward kinematics to solve the physical envelope coordinates to construct the control obstacle function, the spatial anti-collision boundary is transformed into a continuously differentiable constraint equation. This improves the problem that traditional motion planning often uses a discrete collision detection module independent of the generation network, which causes response lag because the discrete search is independent of the generation network.

[0023] 3. In this invention, by calculating the safety compensation scalar based on the obstacle change rate and constraint function, and then applying the minimum obstacle avoidance intervention force as needed when approaching the boundary critical point, the problem of loss of exploration ability caused by the fact that traditional action generation mostly adopts fixed static penalty barriers and the system's generalization space is limited by global forced penalties is improved.

[0024] 4. In this invention, by introducing an inverse dynamics solver, the spatial pose tensor is transformed into torque and current values, thereby separating the gravitational bias and centrifugal force coupling of the physical machinery. This improves the problem that traditional control command issuance links mostly use pure kinematic position issuance, which ignores the inertial characteristics of the equipment and causes deviations in the physical execution trajectory. Attached Figure Description

[0025] Figure 1 The flowchart shows a method for randomly generating VLA action sequences based on a diffusion model proposed in this invention.

[0026] Figure 2 This is a flowchart illustrating the perception and data acquisition process of a VLA action sequence random generation method based on a diffusion model proposed in this invention.

[0027] Figure 3 The flowchart of the feature mapping and control barrier function of the VLA action sequence random generation method based on the diffusion model proposed in this invention is shown below.

[0028] Figure 4 This is a diagram illustrating the iterative solution for action-safe reconstruction of a VLA action sequence random generation method based on a diffusion model proposed in this invention.

[0029] Figure 5 This is an architecture diagram of a VLA action sequence random generation system based on a diffusion model proposed in this invention. Detailed Implementation

[0030] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] Example 1:

[0032] In the first embodiment of the present invention, the present invention provides a method and system for random generation of VLA action sequences based on a diffusion model, such as Figures 1-2 As shown, it includes the following steps:

[0033] S1. Acquire visual images, language commands, joint motion states of the target execution device, and trajectory matrix of moving obstacles;

[0034] Furthermore, in S1, visual images, language commands, the joint motion states of the target execution device, and the trajectory matrix of moving obstacles are acquired, including:

[0035] A continuous video stream within the workspace is acquired using a multi-view visual sensor, and keyframe extraction is performed on the continuous video stream to obtain visual images.

[0036] Receive a text task description sequence from the human-computer interaction terminal, and perform lexical feature encoding processing on the text task description sequence to obtain language instructions;

[0037] Read the encoder data of each joint motor of the target execution device, parse the encoder data of each joint motor to obtain the continuously changing parameters of each joint as the joint motion state;

[0038] Dynamic target detection and pose estimation are performed on the visual image, and the spatial coordinate center point sequence of the moving obstacle within a preset time window is extracted to form the trajectory matrix of the moving obstacle.

[0039] Specifically, a multi-view vision sensor acquires a continuous video stream within the workspace, and performs keyframe extraction on the continuous video stream to obtain visual images. The data flow process involves receiving ambient light signals as input and outputting a discrete keyframe image matrix. A downsampling mathematical model is used to truncate and extract the frame rate from the video stream. Let the extracted visual image matrix be defined as... Its mathematical expression is:

[0040] ;

[0041] in The "belong" symbol represents the set of mathematical elements. Represents the real number field space, This represents the preset number of physical multi-view sensors. Represents the image pixel height parameter. Represents the image pixel width parameter. This represents the number of color channels in an image. In practical terms, it will be... Set to a value of 3 to represent the three primary color channels: red, green, and blue.

[0042] By introducing a keyframe extraction mechanism, the amount of high-frequency temporal redundant data caused by continuous video streams from multiple sensors is reduced. While preserving the complete three-dimensional topological features of the physical working space, the edge-side hardware computing power consumption in the subsequent large model feature alignment process is reduced.

[0043] The system receives a text task description sequence from a human-computer interaction terminal, performs lexical feature encoding on the sequence to obtain language instructions, and receives unstructured string text input, outputting a discrete set of lexical vectors. It maps natural language strings to discrete set equations.

[0044] ;

[0045] in The set of language instructions representing the output of feature encoding. Represents the first term after mapping using a pre-trained dictionary. One word meta-index variable, The length of the maximum total number of terms contained in the text task description sequence.

[0046] Translating human natural language from plain text format into a low-level dimensionality reduction encoding format that machines can directly process provides a clear semantic feature input basis for graph-text cross-attention computation in cross-modal networks.

[0047] The system reads encoder data from each joint motor of the target execution device, parses this data to obtain continuously changing parameters for each joint as its motion state. The data flow involves reading electrical pulse signals from the underlying hardware control bus and outputting the kinematic state vector of the target execution device itself. Let the total degrees of freedom of the robotic arm joints of the target execution device be... The extracted joint motion state vector is defined as :

[0048] ;

[0049] This represents the extracted joint motion state vector. Representing the The absolute physical angle parameters of each rotary joint at the current sampling moment. The total degrees of freedom of the robotic arm joints representing the target execution device. The symbol for the transpose of a vector. represent The real number field space.

[0050] Extracting this vector provides the absolute pose starting point in the current physical world for subsequent diffusion model inverse denoising. Directly introducing the underlying motor encoder physical data severs the information gap between the mathematically predicted action and the physical mechanical limit, preventing the inverse sampling algorithm from generating out-of-limit action commands that violate the current mechanical body boundary.

[0051] Dynamic target detection and pose estimation are performed on the visual image, and the spatial coordinate center point sequence of the moving obstacle within a preset time window is extracted to form the trajectory matrix of the moving obstacle.

[0052] The data flow process involves inputting the aforementioned visual image matrix. Output the obstacle trajectory matrix containing temporal geometric properties. Let the frame length of the preset time window be a scalar. The trajectory matrix of a moving obstacle is defined as follows: :

[0053] ;

[0054] ;

[0055] in Represents the trajectory matrix of moving obstacles. The frame length scalar representing the preset time sliding window. Represents the current physical sampling time point. Represents the center point of the three-dimensional spatial coordinates of the moving obstacle at the current physical sampling time point. This represents the center point of the moving obstacle's three-dimensional spatial coordinates at the previous sampling time point. Represents the moving obstacle at any sampling time The center point of the three-dimensional spatial coordinates, The symbol for the matrix transpose operation; and as well as These represent the absolute coordinate values ​​of the three-dimensional spatial center point in the calibration base coordinate system, respectively, in the three-axis Euclidean space.

[0056] A time sliding window mechanism is established to aggregate the historical displacement of obstacles, providing a continuous and differentiable basic data source for calculating the Lyapunov effectiveness descent conditional partial derivative of the obstacle function in subsequent steps, and preventing the underlying servo actions from causing delayed physical collisions due to the loss of obstacle motion vector characteristics.

[0057] like Figure 3 As shown, S2, the visual image and language command are input into the feature extraction network to obtain the original features, and the semantic mapping matrix is ​​used to map the original features to generate a semantic guidance vector. At the same time, a control obstacle function is constructed based on the joint motion state and the trajectory matrix of the moving obstacle.

[0058] Furthermore, in S2, the visual image and language instructions are input into the feature extraction network to obtain the original features. The semantic mapping matrix is ​​then used to map the original features to generate a semantic guidance vector, including:

[0059] Visual block feature vectors of visual images are extracted by a visual feature encoder in a feature extraction network, and textual latent variable vectors of language instructions are extracted by a language feature encoder in a feature extraction network.

[0060] Perform cross-modal cross-attention fusion computation on the visual block feature vector and the text latent variable vector to extract the original features;

[0061] Obtain the semantic mapping matrix that is aligned with the dimensions of the underlying action manifold space;

[0062] Perform a matrix multiplication and affine transformation operation between the original features and the semantic mapping matrix, and set the vector features after dimension mapping as semantic guide vectors.

[0063] In S2, a control obstacle function is constructed based on the joint motion state and the trajectory matrix of the moving obstacle, including:

[0064] Obtain the physical linkage kinematic model parameters of the target execution device;

[0065] Based on the parameters of the physical linkage kinematic model and the joint motion state, the set of three-dimensional coordinates of the positive physical envelope of each joint of the target execution device is obtained through the positive kinematic solution process of the linkage.

[0066] Extract the instantaneous 3D coordinate center point of the obstacle corresponding to the current timestamp from the trajectory matrix of the moving obstacle;

[0067] Calculate the spatial distance between each coordinate point in the positive physical envelope three-dimensional coordinate set and the instantaneous obstacle three-dimensional coordinate center point, and obtain the pre-calibrated safety boundary tolerance radius value;

[0068] Perform the subtraction operation between the spatial distance value and the safety boundary tolerance radius value, and set the safety boundary distance constraint space equation formed by the subtraction operation as the control obstacle function.

[0069] Specifically, the visual block feature vector of the visual image is extracted by the visual feature encoder in the feature extraction network, and the textual latent variable vector of the language instruction is extracted by the language feature encoder in the feature extraction network. Cross-modal cross-attention fusion computation is then performed on the visual block feature vector and the textual latent variable vector to extract the original features.

[0070] The data input port receives the keyframe image matrix and word sequence set output from the previous step, and the data output port generates a joint representation vector in the latent space. Let the extracted visual block feature vector be... The text latent variable vector is The original features of the fused output after cross-attention calculation are defined as follows: .

[0071] Obtain the semantic mapping matrix aligned with the dimensions of the underlying action manifold space. Perform a matrix multiplication and affine transformation operation between the original features and the semantic mapping matrix, setting the vector features after dimension mapping as semantic guidance vectors. Establish the mathematical formula for the affine transformation from the original features to the target action manifold space:

[0072] ;

[0073] in This represents a semantic guide vector aligned to the action manifold space. This represents the system's pre-defined cross-modal semantic mapping matrix. Represents the original features extracted in the preceding steps. This represents the constant vector of the bias term used to prevent feature zero-point drift.

[0074] The original high-dimensional output features of large visual language models often exhibit a dimensionality mismatch with the underlying servo degrees of freedom of the target execution device. To address this, matrix multiplication and addition operations are introduced to perform spatial basis transformation, reducing the dimensionality of the unstructured large model features and reconstructing them into constraint vectors that the underlying servo system can directly parse. This establishes a communication link between upper-level multimodal cognition and lower-level servo actions.

[0075] Obtain the physical link kinematic model parameters of the target execution device. Based on the physical link kinematic model parameters and joint motion states, obtain the set of three-dimensional coordinates of the positive physical envelope of each joint of the target execution device through the forward kinematics solution process. The data input port receives the current real-time joint angle parameters and the fixed mechanical dimensions of the device, and the data output port generates a set of points in the space occupied by the robotic arm in Cartesian coordinates. Let the obtained set of three-dimensional coordinates of the positive physical envelope be defined as... Extract the coordinates of a single physical envelope point within the set as follows: .

[0076] Extract the instantaneous 3D coordinate center point of the obstacle corresponding to the current timestamp from the trajectory matrix of the moving obstacle. Calculate the spatial distance between each coordinate point in the forward physical envelope 3D coordinate set and the instantaneous 3D coordinate center point of the obstacle, and obtain the pre-calibrated safety boundary tolerance radius value. Perform the subtraction operation between the spatial distance value and the safety boundary tolerance radius value, and set the safety boundary distance constraint space equation formed by the subtraction operation as the control obstacle function.

[0077] Let the center point of the instantaneous obstacle's three-dimensional coordinates be... Establish the control barrier function equation characterizing the physical collision avoidance boundary:

[0078] ;

[0079] in This represents the control barrier function being constructed. Represents the coordinates of points in the physical envelope space within the set of three-dimensional coordinates of the positive physical envelope. This represents the center point of the obstacle's 3D coordinates at the current timestamp. This represents the Euclidean distance between the coordinates of a point in the physical envelope space and the instantaneous 3D coordinate center point of the obstacle. This represents the calibrated and fixed safety boundary tolerance radius value.

[0080] The physical envelope boundary of the robotic arm is obtained based on the forward kinematics solution, and a distance constraint equation is established in conjunction with the safety tolerance radius. This step transforms the abstract dynamic collision avoidance safety requirements into an absolutely continuous and differentiable mathematical constraint surface, laying the calculus foundation for calculating the normal vector penalty gradient far from the hazard source in the inverse denoising iteration step, and ensuring that the generated final action sequence cannot exceed the mechanical and physical safety limits.

[0081] S3. Sample the initial noise sequence to start the reverse denoising iteration, and in the current iteration step, calculate the partial derivative of the control obstacle function with respect to the current action state, and extract the normal vector gradient matrix. The normal vector gradient matrix indicates the steepest ascent direction pointing to the boundary of the safe area.

[0082] Furthermore, in S3, the initial noise sequence is sampled to initiate the inverse denoising iteration. In the current iteration step, the partial derivative of the control barrier function with respect to the current action state is calculated, and the normal vector gradient matrix is ​​extracted. The normal vector gradient matrix indicates the steepest ascent direction pointing towards the boundary of the safe region, including:

[0083] Gaussian distribution sampling is performed in the action space to obtain the initial noise sequence and then the inverse denoising iteration is started.

[0084] The current action state sampled in the current iteration step of the inverse denoising iteration is represented as an action state tensor;

[0085] Establish a system of multivariate partial differential equations for the control barrier function with respect to the action state tensor;

[0086] The Jacobian matrix is ​​solved by performing partial derivative numerical calculation on the system of multivariable partial differential equations, and the gradient distribution data of the direction vector of the Jacobian matrix is ​​extracted and set as the normal vector gradient matrix.

[0087] Specifically, Gaussian distribution sampling is performed within the action space to obtain an initial noise sequence, and inverse denoising iteration is initiated. The data input acquires preset high-dimensional action space dimension parameters, and the output generates a random mathematical tensor following a standard normal distribution. The Gaussian sampling initialization mathematical model is established as follows:

[0088] ;

[0089] in This represents the maximum set number of iterations. The initial noise sequence at that time, This represents the standard normal distribution operation identifier, where 0 represents the distribution's expected value and zero mean constant vector. This represents the unit diagonal matrix corresponding to the dimension of the underlying action manifold space. Gaussian distribution sampling provides the initial boundary for the Markov chain numerical solution in the reverse diffusion reconstruction process. By introducing a pure noise starting point mechanism, the rigid call path to manually calibrated expert demonstration data is severed, providing a starting base with full ergonomic exploration characteristics for action generation in complex and variable operating environments, thus avoiding the trap of the underlying large model inference falling into local non-physical minima.

[0090] To calculate the partial derivative of the control obstacle function with respect to the current action state, the system represents the current action state sampled in the current iteration step of the inverse denoising iteration as an action state tensor, and then establishes a system of multivariate partial differential equations for the control obstacle function with respect to this action state tensor. The system of multivariate partial differential equations is then solved by numerical calculation of the partial derivatives to obtain the Jacobian matrix, and the gradient distribution data of the direction vector of the Jacobian matrix is ​​extracted and set as the normal vector gradient matrix.

[0091] The data input receives the control obstacle function equation constructed by previous dimensionality reduction and alignment, as well as the action state extracted from the current timestamp node. The output generates a high-dimensional gradient matrix tensor with absolute spatial direction pointing attributes. The index variable for the current iteration step in inverse denoising is set as... Establish the Jacobian matrix to solve the mathematical equation:

[0092] ;

[0093] in This represents the gradient matrix of the extracted normal vector. This represents a multivariate partial differential operator performed on the action-state space tensor. Represents the inverse denoising iteration at the 1st The current action state tensor is obtained by solving the step-by-step numerical solution. This represents the control obstacle function constraint equation established in the preceding operation phase and substituted with the motion manifold space variables.

[0094] The scalar form of the physical safety distance constraint threshold is reconstructed into a high-dimensional vector derivative feature within the manifold space. The generated normal vector gradient matrix explicitly points to the safe region space in the diametric manifold section where the fastest escape from the physical envelope surface interference of the moving obstacle occurs; that is, the normal vector gradient matrix indicates the fastest ascent direction pointing to the boundary of the safe region. The static obstacle avoidance judgment condition is directly translated into a repulsive gradient operator that can participate in the calculation of differential equations in the denoising neural network. This establishes an algebraic topological level mandatory avoidance basis for subsequent implementation of orthogonal null space projection operator mapping, preventing the low-level motion response lag defect caused by the traversal node search calculation in conventional external collision detection engines.

[0095] S4. Construct a null space projection operator based on the normal vector gradient matrix, and use the null space projection operator to orthogonally map the task score gradient generated by the denoising network based on the semantic guidance vector to extract the safe task gradient.

[0096] Furthermore, in S4, a null space projection operator is constructed based on the normal vector gradient matrix. This null space projection operator is then used to orthogonally map the task score gradient generated by the denoising network based on the semantically guided vector, extracting the safety task gradient, including:

[0097] Perform a matrix transpose operation on the normal vector gradient matrix to obtain the transposed gradient matrix, and calculate the dot product of the normal vector gradient matrix and the transposed gradient matrix.

[0098] Perform matrix inversion on the result of the matrix dot product operation to obtain the generalized inverse matrix components of the corresponding dimension, and construct the null space projection operator by subtracting the normal gradient matrix from the standard identity matrix and multiplying the generalized inverse matrix components.

[0099] The current action state and semantic guidance vector are input into a parameterized denoising network to infer and output the task score gradient.

[0100] Perform a matrix multiplication mapping operation between the task score gradient and the null space projection operator, remove the action tensor components in the task score gradient that have tensor features in the direction parallel to the normal vector gradient matrix, and output the retained orthogonalized action tensor feature set as the safe task gradient.

[0101] Specifically, the transpose operation is performed on the normal vector gradient matrix to obtain the transposed gradient matrix. The dot product of the normal vector gradient matrix and the transposed gradient matrix is ​​then calculated. The matrix inversion operation is performed on the dot product result to obtain the generalized inverse matrix component of the corresponding dimension. The null space projection operator is constructed by subtracting the product of the normal vector gradient matrix and the generalized inverse matrix component from the standard identity matrix. The data input receives the high-dimensional gradient tensor of the normal vector calculated in the previous steps, and the output generates a projection operator matrix with directional filtering properties. The equations for solving the projection operator are established as follows:

[0102] ;

[0103] in Represents the generated null-space projection operator. This represents a standard unit diagonal matrix with the same dimensions as the underlying action manifold space. This represents the gradient matrix of the obtained normal vector. Represents the transpose gradient matrix. This represents the components of the generalized inverse matrix obtained through dot product and inversion operations.

[0104] The single-dimensional repulsion vector extension is transformed into a tensor filtering constraint boundary within the global motion space. The full motion tensor space of the underlying servo system is reconstructed and divided into a collision hazard tensor quantum space and an absolutely safe tensor quantum space.

[0105] The current action state and semantic guidance vector are input into a parameterized denoising network for inference, outputting the task score gradient. The data flow process involves extracting the action sequence tensor and multimodal representation vector of the current inverse denoising time step, and outputting the original update vector with semantic guidance rules through network forward propagation. The inference equation of the denoising network is established as follows:

[0106] ;

[0107] in The gradient of the task score representing the inference output. This represents a denoising network loaded with pre-trained weights. The action state input tensor represents the action state sampled at the current time iteration step. Represents a semantic guidance vector. This represents the current time step index scalar for inverse denoising. It extracts the native job execution features of the large model under unrestricted physical conditions, obtaining undisturbed action generation task coherent attribute data.

[0108] The matrix multiplication mapping operation between the task score gradient and the null space projection operator is performed. Action tensor components with parallel directions to the normal vector gradient matrix are extracted from the task score gradient. The retained orthogonalized action tensor feature set is output as the safe task gradient. The input data includes the original job execution features and the projection operator matrix. The output data generates corrected gradient features after removing illegal collision components. An algebraic mapping operation equation is constructed:

[0109] ;

[0110] in This represents the security task gradient extracted after mapping processing. Represents the null space projection operator constructed in the preceding sequence. This represents the gradient of task scores.

[0111] High-dimensional algebraic projection calculations are implemented to forcibly eliminate all illegal displacement trends pointing towards the physical envelope of obstacles in the original operational features of the large model. Since the gradient of the safety task and the gradient matrix of the normal vector are strictly orthogonal and their inner product is zero in Euclidean geometric space, the root cause of gradient adversarial between semantic task execution and physical obstacle avoidance is completely eliminated. This prevents control lag defects caused by relying on external collision detectors to perform brute-force interception. A highest-level collision avoidance execution layer is established at the bottom layer of inverse calculus solution, preventing the control equipment from falling into a deadlock due to high-frequency trajectory oscillations caused by feature-in-the-direction conflicts.

[0112] like Figure 4 As shown, S5 calculates the safety compensation scalar based on the control obstacle function, superimposes the normal vector gradient matrix weighted by the safety compensation scalar onto the safety task gradient, obtains the updated action gradient to update the current action state, iterates in reverse denoising until the stopping condition is met, and outputs the target action sequence.

[0113] Furthermore, in S5, the safety compensation scalar is calculated based on the control obstacle function, including:

[0114] The time-domain obstacle change rate characteristic value is obtained by performing a computational operation on the time-dimensional partial derivative of the control obstacle function;

[0115] Obtain a strictly monotonically increasing expansion function to constrain the system state convergence boundary, and input the control barrier function into the strictly monotonically increasing expansion function to calculate and obtain the nonlinear state convergence characteristic term;

[0116] Perform a reverse summation and merging operation on the eigenvalues ​​of the time-domain barrier rate of change and the nonlinear state convergence eigenterms to obtain the reverse summation result;

[0117] Calculate the vector norm of the gradient matrix of the normal vector and perform a squaring operation to obtain the squared value of the gradient norm. Perform a division operation by dividing the summation result by the squared value of the gradient norm to extract the safety compensation scalar that satisfies the collision avoidance constraint conditions.

[0118] In S5, the scalar-weighted normal vector gradient matrix with safety compensation is superimposed on the safety task gradient to obtain the updated action gradient to update the current action state. The reverse denoising iteration continues until the stopping condition is met, outputting the target action sequence, including:

[0119] Perform a scalar multiplication tensor amplification operation between the safety compensation scalar and the normal vector gradient matrix to obtain the obstacle avoidance compensation control vector;

[0120] The obstacle avoidance compensation control vector and the safety task gradient execution vector are added together item by item to obtain the updated action gradient;

[0121] A single-step numerical reconstruction operation is performed on the current action state using the updated action gradient and the randomly injected hot-annealed Gaussian noise tensor.

[0122] Determine whether the number of iterations executed in the reverse denoising iteration has reached the preset maximum step limit. If the preset maximum step limit has not been reached, perform the single-step numerical reconstruction operation of the current action state in a loop. If the preset maximum step limit is reached, end the iteration process and set the action state tensor output by the last reconstruction step as the target action sequence.

[0123] Specifically, the time-domain obstacle rate of change eigenvalue is obtained by calculating the time-domain partial derivative of the control obstacle function. A strictly monotonically increasing expansion function is obtained to constrain the system state convergence boundary. The control obstacle function is then input into this function to calculate the nonlinear state convergence eigenvalue. The time-domain obstacle rate of change eigenvalue and the nonlinear state convergence eigenvalue are combined using a reverse summation operation to obtain the reverse summation result. The vector norm of the normal vector gradient matrix is ​​calculated and squared to obtain the squared gradient norm value. The reverse summation result is then divided by the squared gradient norm value to extract the safety compensation scalar value that satisfies the collision avoidance constraints.

[0124] The data input receives the previously calculated control barrier function and normal gradient matrix, while the data output generates a scalar value representing the intensity of physical intervention. A scalar solution equation for safety compensation based on the Lyapunov effectiveness decline condition is established:

[0125] ;

[0126] in This represents the extracted security compensation scalar. This represents the maximum value operator. 0 represents the reference constant zero. The partial derivative of the control barrier function in the time dimension represents the characteristic value of the rate of change of the barrier in the time domain. This represents a strictly monotonically increasing extended function. This represents the control barrier function. This represents the convergence characteristic term of the nonlinear state. The gradient matrix represents the normal vector. The squared 2-norm of the gradient matrix representing the normal vector is the value of the gradient norm squared.

[0127] The activation condition for obstacle avoidance is strictly defined at the critical state boundary where a collision is about to occur on the physical envelope. The minimum intervention amplitude that satisfies safe convergence is calculated by relying on the nonlinear mapping characteristics of a strictly monotonically increasing extension function, preventing the loss of the model's generalization and exploration capabilities under normal operating conditions due to the introduction of a fixed static physical barrier.

[0128] The obstacle avoidance compensation control vector is obtained by performing a scalar multiplication tensor amplification operation on the safety compensation scalar and the normal vector gradient matrix. The obstacle avoidance compensation control vector and the safety task gradient are then combined term-by-term to obtain the updated action gradient. A single-step numerical reconstruction operation is performed on the current action state using the updated action gradient and a randomly injected hot-annealed Gaussian noise tensor. It is determined whether the number of iterations in the inverse denoising iteration has reached a preset maximum step limit. If not, the single-step numerical reconstruction operation on the current action state is performed repeatedly. The iteration process ends when the preset maximum step limit is reached, and the action state tensor output from the last reconstruction step is set as the target action sequence.

[0129] The data input receives the safety compensation scalar, the normal vector gradient matrix, and the safety task gradient. The data output generates a set of continuous physical execution actions that have undergone noise reduction and convergence. The overall control score vector synthesis equation and the discrete Langevin dynamics inverse update equation are constructed as follows:

[0130] ;

[0131] ;

[0132] in This represents updating the action gradient. This represents the safety task gradient extracted after the preceding steps remove illegal collision components. Represents a safety compensation scalar. The gradient matrix represents the normal vector. This represents the action state tensor of the next time step after a single-step numerical reconstruction operation. The tensor representing the current action state at the current time step. This represents the set denoising integration step size constant. This represents the randomly injected thermally annealed Gaussian noise tensor.

[0133] A unified control logic for physical obstacle avoidance and semantic task execution is established. The safety task gradient and obstacle avoidance compensation control vector retain orthogonal and independent properties in algebraic geometric space. The overall control manifold gradient directly guides the denoising network to avoid the physical envelope of moving obstacles and slide along the tangent direction in the multidimensional feature space. A cycle-blocking stopping condition based on Markov chains is established to complete the mathematical decoding transformation from pure Gaussian noise to deterministic collision-free physical operation actions.

[0134] S6. Convert the target action sequence into drive instructions to control the target execution device;

[0135] Furthermore, in S6, the target action sequence is converted into drive instructions to control the target execution device, including:

[0136] Extract the desired spatial pose node dataset of each joint of the target execution device at each time step from the target action sequence with time series characteristics according to the preset control cycle.

[0137] Input the desired spatial pose node dataset of each joint into the preset inverse dynamics solver to convert the pose tensor data into the torque target numerical sequence and current target numerical sequence of the corresponding joint servo motor.

[0138] Pack the electrical control signals containing the target torque value sequence and the target current value sequence into a drive command;

[0139] The drive commands are sent to the underlying hardware microcontroller of the target execution device through a real-time communication bus to drive the joint motors of each joint to perform displacement and work operations in a closed loop.

[0140] Specifically, the expected spatial pose node dataset of each joint of the target execution device at each time step is extracted sequentially from the target action sequence with time-series characteristics according to a preset control cycle.

[0141] The data input terminal receives the target action sequence matrix after inverse denoising and convergence, and the data output terminal generates discrete kinematic variables that conform to the underlying hardware clock cycle. Let the preset control period of the target device be... The first action extracted from the target action sequence The dataset of joint desired spatial pose nodes at each time step contains desired angle vectors. Desired angular velocity vector and the desired angular acceleration vector .

[0142] The macroscopic continuous trajectory inferred from the diffusion model is sliced ​​in the time domain according to the controller sampling frequency to establish a data synchronization benchmark between the upper-layer algorithm and the lower-layer driver, thus cutting off the instruction backlog or execution gap caused by clock mismatch.

[0143] Input the desired spatial pose node dataset of each joint into the preset inverse dynamics solver to convert the pose tensor data into the torque target numerical sequence and current target numerical sequence of the corresponding joint servo motor.

[0144] The data receives the aforementioned set of kinematic variables and outputs the driving torque and electrical parameters corresponding to the bottom layer of the joint. The inverse dynamic torque solution equation and the corresponding current conversion equation based on the rigid body linkage mechanism are established:

[0145] ;

[0146] ;

[0147] in The system inertia matrix represents the target execution device in the desired pose in the corresponding space. The characteristic matrix representing the coupling of Coriolis force and centrifugal force generated during the motion of the representative mechanism. This represents the gravity compensation vector generated by the influence of gravity on the equipment. This represents the sequence of target torque values ​​for the corresponding servo motor output from the solution. It represents the inverse matrix of the inherent torque constant matrix of the motor. This represents the sequence of target current values ​​obtained from torque conversion.

[0148] The physical properties of the equipment, including mass and inertia distribution data, are used for dynamic compensation calculations. By eliminating the coupling interference of gravity bias and centrifugal force during high-speed multi-joint linkage, the displacement intention in pure geometric space is accurately translated into the physical torque and electrical control variables required to drive the main shaft of the motor.

[0149] The electrical control signals containing the target torque and current numerical sequences are packaged and configured as drive commands. The data flow process involves extracting the underlying floating-point calculation values ​​obtained from previous calculations and encapsulating them into a standard data frame structure according to the specified industrial bus communication protocol. This provides the servo drive module with control messages that can be directly mapped to its internal memory registers, establishing the media conversion method from the algorithm layer to the physical communication layer.

[0150] The drive commands are sent to the underlying hardware microcontroller of the target execution device through a real-time communication bus to drive the joint motors of each joint to perform displacement and work operations in a closed loop.

[0151] After receiving the message command, the underlying hardware microcontroller uses the current target value sequence as the input reference to adjust the duty cycle output of the inverter power device, thereby driving the servo motor rotor to generate continuous physical angular displacement.

[0152] It completes the entire closed-loop transformation from multimodal representation semantic understanding to physical entity work implementation. It utilizes the hardware-level high-frequency closed-loop feedback mechanism of the underlying microcontroller to resist random physical friction and load cutting disturbances in the working environment, ensuring that the final output action strictly follows the decoupled safety planning trajectory.

[0153] Example 2:

[0154] In human-machine collaborative assembly scenarios in flexible manufacturing workshops, collaborative robotic arms need to perform continuous grasping operations based on natural language instructions, and the work space contains dynamically moving obstacles such as randomly moving AGVs. Existing technologies using traditional visual language action models and diffusion strategies to control the robotic arm result in direct time-varying coupling between the semantic features of the denoising network (approaching the task target) and the repulsion features (avoiding dynamic obstacles) in the same high-dimensional action space. Because obstacles continuously generate physical spatial displacement, this feature coupling leads to opposing gradient conflicts between "task execution" and "obstacle avoidance" during the inverse solution iteration. This causes the underlying servo system of the robotic arm to fall into a dynamic spatiotemporal deadlock, generating extremely oscillating and discontinuous non-physical motion trajectories. Ultimately, this results in physical collisions between the robotic arm and moving obstacles at trajectory intersections, or triggers a forced protective shutdown of the servo driver, causing a complete interruption of the assembly line operation. To solve these problems, this invention provides a VLA action sequence random generation system based on a diffusion model, the structure of which is as follows: Figure 5 As shown. The specific implementation process of this system is as follows:

[0155] A complete technical loop, from multimodal perception to physical entity execution, is constructed through the underlying data flow of each functional module. The state data acquisition module provides heterogeneous and multi-source physical perception benchmarks for subsequent inference. The semantic mapping and obstacle construction module maps unstructured visual language input into guiding vectors aligned with the underlying control dimensions and constructs collision avoidance safety boundary equations using time-varying trajectory features. The noise sampling and gradient extraction module initiates a Markov inverse denoising process, transforming the spatial scalar boundary into a high-dimensional normal vector repulsion vector pointing to the fastest escape from the danger zone. The core null space orthogonal mapping module performs algebraic geometric mapping calculations, forcibly removing illegal displacement tensor components parallel to the obstacle avoidance vector in the task inference results, cutting off feature adversarial at the root of mathematical derivation. The action state reconstruction module solves the control scalar that satisfies the descent convergence boundary and performs tensor synthesis to reconstruct a safe action sequence with continuous physical differentiability. Finally, the servo drive control module parses the high-dimensional mathematical tensor into motor closed-loop operation signals. The coordinated operation of each module eliminates time-varying coupling interference when the execution device faces dynamic obstacles, avoids the motion control layer from falling into a spatiotemporal deadlock due to oscillation and resampling, and ensures the rigid physical boundary safety of continuous collaborative operation.

[0156] Furthermore, in order to expand the generalization capability of the system of the present invention under different computing power platforms and specific operating scenarios, for some core modules in the above embodiments, those skilled in the art can adopt the following equivalent replacement schemes without departing from the core concept of the present invention:

[0157] Specifically, in the semantic mapping and obstacle construction stages, the control obstacle function is not limited to Euclidean distance calculation. When dealing with dense moving obstacle scenarios, a composite obstacle function can be formed by constructing a union of control obstacle functions for multiple obstacles and combining them using a smooth maximum function; alternatively, the repulsive gradient direction of an artificial potential field function can be used to equivalently replace the normal vector gradient matrix of the control obstacle function, which can also achieve the extraction of the repulsive vector.

[0158] Specifically, in the null space orthogonal mapping stage, the techniques for removing the features of the parallel direction tensors are not limited to explicit null space projection operator matrix multiplication. When there is redundancy in the underlying servo degrees of freedom, an equivalent replacement can be achieved by introducing a task priority solver based on quadratic programming, or by constructing a projection operator using a weighted generalized inverse matrix adjusted by the state weight matrix, thereby achieving the same goal of decoupling the orthogonalized action tensor features.

[0159] Specifically, in the noise sampling and gradient extraction stages, the inverse denoising iteration is not limited to the random sampling mechanism of the denoising diffusion probability model. To meet the real-time control requirements of extremely high frequencies, an implicit denoising diffusion model deterministic sampling strategy or a consistency model step-by-step sampling mechanism can be introduced as equivalent replacements, thereby effectively compressing the iteration cycle of calculus solutions while ensuring the topological continuity of the action sequence space.

[0160] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for randomly generating VLA action sequences based on a diffusion model, characterized in that, Includes the following steps: S1. Acquire visual images, language commands, joint motion states of the target execution device, and trajectory matrix of moving obstacles; S2. Input the visual image and the language instruction into the feature extraction network to obtain the original features, use the semantic mapping matrix to map the original features to generate a semantic guidance vector, and at the same time construct a control obstacle function based on the joint motion state and the trajectory matrix of the moving obstacle. S3. Sample the initial noise sequence to start the reverse denoising iteration, and in the current iteration step, calculate the partial derivative of the control obstacle function with respect to the current action state, and extract the normal vector gradient matrix. The normal vector gradient matrix indicates the steepest ascent direction pointing to the boundary of the safe area. S4. Construct a null space projection operator based on the normal vector gradient matrix, and use the null space projection operator to orthogonally map the task score gradient generated by the denoising network based on the semantic guidance vector to extract the safe task gradient. S5. Calculate the safety compensation scalar based on the control obstacle function, superimpose the normal vector gradient matrix weighted by the safety compensation scalar onto the safety task gradient, obtain the update action gradient to update the current action state, repeat the inverse denoising iteration until the stopping condition is met, and output the target action sequence. S6. Convert the target action sequence into drive instructions to control the target execution device.

2. The method for randomly generating VLA action sequences based on a diffusion model according to claim 1, characterized in that, In S1, acquiring visual images, language commands, joint motion states of the target execution device, and trajectory matrix of moving obstacles includes: A continuous video stream within the workspace is acquired using a multi-view visual sensor, and a keyframe extraction operation is performed on the continuous video stream to obtain the visual image. Receive a text task description sequence from a human-computer interaction terminal, and perform lexical feature encoding processing on the text task description sequence to obtain the language instructions; Read the encoder data of each joint motor of the target execution device, parse the encoder data of each joint motor to obtain the continuously changing parameters of each joint as the joint motion state; Dynamic target detection and pose estimation are performed on the visual image, and the spatial coordinate center point sequence of the moving obstacle within a preset time window is extracted to form the trajectory matrix of the moving obstacle.

3. The method for randomly generating VLA action sequences based on a diffusion model according to claim 1, characterized in that, In S2, the step of obtaining original features by inputting the visual image and the language instruction into the feature extraction network, and generating a semantic guidance vector by mapping the original features using a semantic mapping matrix, includes: The visual block feature vector of the visual image is extracted by the visual feature encoder in the feature extraction network, and the text latent variable vector of the language instruction is extracted by the language feature encoder in the feature extraction network. Perform cross-modal cross-attention fusion computation on the visual block feature vector and the text latent variable vector to extract the original features; Obtain the semantic mapping matrix configured to align with the dimensions of the underlying action manifold space; Perform a matrix multiplication and affine transformation operation between the original features and the semantic mapping matrix, and set the vector features after dimension mapping as the semantic guiding vector.

4. The method for randomly generating VLA action sequences based on a diffusion model according to claim 1, characterized in that, In S2, the construction of the obstacle control function based on the joint motion state and the trajectory matrix of the moving obstacle includes: Obtain the physical linkage kinematic model parameters of the target execution device; Based on the physical link kinematic model parameters and the joint motion state, the positive physical envelope three-dimensional coordinate set of each joint of the target execution device is obtained through the link positive kinematics solution process; Extract the instantaneous 3D coordinate center point of the obstacle corresponding to the current timestamp from the trajectory matrix of the moving obstacle; Calculate the spatial distance between each coordinate point in the positive physical envelope three-dimensional coordinate set and the center point of the instantaneous obstacle three-dimensional coordinate set, and obtain the pre-calibrated safety boundary tolerance radius value; Perform the difference operation between the spatial distance value and the safety boundary tolerance radius value, and set the safety boundary distance constraint space equation formed by the difference operation as the control obstacle function.

5. The method for randomly generating VLA action sequences based on a diffusion model according to claim 1, characterized in that, In S3, the initial noise sequence is sampled to initiate a reverse denoising iteration. In the current iteration step, the partial derivative of the control barrier function with respect to the current action state is calculated, and the normal vector gradient matrix is ​​extracted. This normal vector gradient matrix indicates the steepest ascent direction pointing towards the boundary of the safe zone, including: The initial noise sequence is obtained by performing Gaussian distribution sampling within the action space, and the inverse denoising iteration is initiated. The current action state sampled in the current iteration step by the inverse denoising iteration is represented as an action state tensor; Establish a system of multivariate partial differential equations for the control obstacle function with respect to the action state tensor; The Jacobian matrix is ​​solved by performing partial derivative numerical calculation on the multivariate partial differential equation system, and the gradient distribution data of the direction vector of the Jacobian matrix is ​​extracted and set as the normal vector gradient matrix.

6. The method for randomly generating VLA action sequences based on a diffusion model according to claim 1, characterized in that, In S4, the step of constructing a null space projection operator based on the normal vector gradient matrix, and using the null space projection operator to orthogonally map the task score gradient generated by the denoising network based on the semantic guidance vector to extract the safety task gradient includes: Perform a matrix transpose operation on the normal vector gradient matrix to obtain the transposed gradient matrix, and calculate the matrix dot product of the normal vector gradient matrix and the transposed gradient matrix; Perform matrix inversion on the result of the matrix dot product operation to obtain the generalized inverse matrix component of the corresponding dimension, and construct the null space projection operator by subtracting the product of the normal vector gradient matrix and the generalized inverse matrix component from the standard identity matrix; The current action state is input into the denoising network parameterized by the semantic guidance vector to infer and output the task score gradient. Perform a matrix multiplication mapping operation between the task score gradient and the null space projection operator, remove the action tensor components in the task score gradient that have parallel direction tensor features with the normal vector gradient matrix, and output the retained orthogonalized action tensor feature set as the safe task gradient.

7. The method for randomly generating VLA action sequences based on a diffusion model according to claim 1, characterized in that, In S5, the step of calculating the safety compensation scalar based on the control obstacle function includes: The time-domain obstacle change rate characteristic value is obtained by performing a time-dimensional partial derivative calculation on the control obstacle function; Obtain a strictly monotonically increasing expansion function to constrain the system state convergence boundary, and input the control barrier function into the strictly monotonically increasing expansion function to calculate and obtain the nonlinear state convergence characteristic term; Perform a reverse summation and merging operation on the time-domain barrier change rate characteristic value and the nonlinear state convergence characteristic term to obtain the reverse summation result; The vector norm of the gradient matrix of the normal vector is calculated and the square operation is performed to obtain the squared value of the gradient norm. The inverse summation result is divided by the squared value of the gradient norm to extract the safety compensation scalar whose value satisfies the anti-collision constraint condition.

8. The method for randomly generating VLA action sequences based on a diffusion model according to claim 1, characterized in that, In S5, the step of superimposing the normal vector gradient matrix, which is weighted by the safety compensation scalar, onto the safety task gradient to obtain the updated action gradient to update the current action state, and repeating the inverse denoising iteration until the stopping condition is met, and outputting the target action sequence, includes: Perform a scalar multiplication tensor amplification operation between the safety compensation scalar and the normal vector gradient matrix to obtain the obstacle avoidance compensation control vector; The obstacle avoidance compensation control vector and the safety task gradient execution vector are added together item by item to obtain the updated action gradient; A single-step numerical reconstruction operation is performed on the current action state using the updated action gradient and the randomly injected hot-annealed Gaussian noise tensor. Determine whether the number of iterations executed in the reverse denoising iteration has reached a preset maximum step limit. If the preset maximum step limit has not been reached, perform the single-step numerical reconstruction operation of the current action state in a loop. If the preset maximum step limit is reached, end the iteration process and set the action state tensor output by the last reconstruction step as the target action sequence.

9. The method for randomly generating VLA action sequences based on a diffusion model according to claim 1, characterized in that, In S6, converting the target action sequence into drive instructions to control the target execution device includes: The target action sequence with time-series characteristics is extracted sequentially according to a preset control cycle to obtain the expected spatial pose node dataset of each joint of the target execution device at each time step. The desired spatial pose node datasets of each joint are input into a preset inverse dynamics solver to convert the pose tensor data into the torque target numerical sequence and current target numerical sequence of the corresponding joint servo motor. The electrical control signals containing the target torque value sequence and the target current value sequence are packaged and set as the drive command; The drive commands are sent to the underlying hardware microcontroller of the target execution device via a real-time communication bus to drive each joint motor to perform displacement and work operations in a closed loop.

10. A VLA action sequence random generation system based on a diffusion model, characterized in that, A method for randomly generating VLA action sequences based on a diffusion model, as described in any one of claims 1-9, comprises the following modules: The status data acquisition module is configured to acquire visual images, language commands, joint motion states of the target execution device, and trajectory matrices of moving obstacles. The semantic mapping and obstacle construction module is configured to obtain original features by combining the visual image with the language instruction input feature extraction network, generate a semantic guidance vector by mapping the original features using a semantic mapping matrix, and construct a control obstacle function based on the joint motion state and the trajectory matrix of the moving obstacle. The noise sampling and gradient extraction module is configured to sample the initial noise sequence to start the inverse denoising iteration, and in the current iteration step, calculate the partial derivative of the control barrier function with respect to the current action state to extract the normal vector gradient matrix, which indicates the steepest ascent direction pointing to the boundary of the safe area; The null space orthogonal mapping module is configured to construct a null space projection operator based on the normal vector gradient matrix, and use the null space projection operator to orthogonally map the task score gradient generated by the denoising network based on the semantic guidance vector to extract the safe task gradient. The action state reconstruction module is configured to calculate the safety compensation scalar based on the control obstacle function, superimpose the normal vector gradient matrix weighted by the safety compensation scalar onto the safety task gradient, obtain the updated action gradient to update the current action state, loop the inverse denoising iteration until the stopping condition is met, and output the target action sequence. The servo drive control module is configured to convert the target action sequence into drive instructions to control the target execution device.

Images