Method and system for generating a goods picking and placing strategy

Abstract

The application provides a kind of goods grabbing and placing strategy generation method and system, method includes the sampling processing to original video data, obtains continuous video frame sequence;Video frame sequence is encoded, and the discrete visual vocabulary sequence is generated;The time sequence dependence modeling of visual vocabulary sequence is carried out to learn the continuous change rule of object action in the video content represented by visual vocabulary sequence;Based on the visual vocabulary corresponding to the current state of the goods to be operated and the visual vocabulary corresponding to the expected target state, and using the continuous change rule of object action, the latent action vector representing the action execution sequence required for the transition from the current state to the target state is generated;The real-time physical state information of the robot obtained is mapped with the latent action vector, to generate the control instruction for driving the robotic arm to perform goods grabbing or placing operation, effectively reduces the dependence on a large amount of manually labeled data, and improves the self-adaptation and generalization ability of the model to unseen scenarios.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and robot control technology, and in particular to a method and system for generating a goods picking and placing strategy. Background Technology

[0002] With the rapid development of e-commerce and intelligent manufacturing, the demand for picking efficiency in warehousing and logistics is increasing, and robotic arm picking systems have gradually replaced manual labor as the mainstream solution. Currently, the mainstream motion planning and control technologies for picking and handling robots are mainly divided into rule-based programming, learning from demonstration, and vision recognition-based grasping methods.

[0003] However, rule-based programming relies on preset paths and fixed templates, lacking the ability to generalize to changes in the position and posture of objects; demonstration-based learning requires a large amount of manually labeled expert demonstration data, which is costly in terms of data collection and labeling; and visual recognition-based methods rely on accurate object detection and pose estimation, which are less robust in complex scenarios such as changes in lighting, occlusion, or similar appearances, making it difficult to meet the high reliability requirements of actual business. Summary of the Invention

[0004] This invention provides a method and system for generating goods picking and placement strategies, in order to solve the technical problems of high data costs and poor scenario adaptability caused by reliance on manual annotation and rule programming in the prior art.

[0005] On one hand, the present invention provides a method for generating a goods picking and placing strategy, comprising:

[0006] Obtain raw video data of goods handling;

[0007] The original video data is sampled and processed to obtain a continuous sequence of video frames;

[0008] The video frame sequence is encoded to generate a discrete visual vocabulary sequence;

[0009] Temporal dependency modeling is performed on visual word sequences to learn the continuous change patterns of object actions in video content represented by the visual word sequences.

[0010] Based on the visual vocabulary corresponding to the current state of the goods to be operated and the visual vocabulary corresponding to the expected target state, and by utilizing the continuous change pattern of the object's actions, a potential action vector representing the action execution sequence required to transition from the current state to the target state is generated.

[0011] By mapping the potential motion vectors to the acquired real-time physical state information of the robot, control commands are generated to drive the robot's robotic arm to perform goods grasping and placing operations.

[0012] On the other hand, the present invention also provides a system for generating a goods picking and placing strategy, comprising:

[0013] The data acquisition module is used to acquire raw video data of goods handling.

[0014] The data sampling module is used to sample and process the raw video data to obtain a continuous sequence of video frames;

[0015] The visual vocabulary module is used to encode video frame sequences to generate discretized visual vocabulary sequences.

[0016] The temporal dependency module is used to perform temporal dependency modeling on visual word sequences in order to learn the continuous change pattern of object actions in the video content represented by the visual word sequences.

[0017] The latent action module is used to generate a latent action vector that represents the action execution sequence required to transition from the current state to the target state, based on the visual vocabulary corresponding to the current state of the goods to be operated and the visual vocabulary corresponding to the desired target state, and by utilizing the continuous change pattern of the object's actions.

[0018] The instruction generation module is used to map potential motion vectors to the acquired real-time physical state information of the robot, and generate control instructions to drive the robot's robotic arm to perform goods grasping and placing operations.

[0019] The present invention provides a method and system for generating goods grasping and placement strategies. This method automatically learns the continuous change patterns of object movements from unlabeled raw video, and uses these patterns in conjunction with the target state to generate abstract potential action vectors. These vectors are then mapped to precise control commands that the robot can execute. This method effectively reduces reliance on large amounts of manually labeled data, improves the model's adaptability and generalization ability to unseen scenes, and ensures the continuity and physical rationality of the generated actions through temporal modeling. Therefore, it can achieve highly robust and efficient automated grasping and placement operations in complex dynamic environments. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the method for generating a goods picking and placing strategy according to an embodiment of the present invention.

[0022] Figure 2This is a schematic diagram of the structure of the product grabbing and placement strategy generation system provided in the embodiment of the present invention;

[0023] Figure 3 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0025] Figure 1 This is a flowchart illustrating the method for generating a goods picking and placing strategy according to an embodiment of the present invention.

[0026] See Figure 1 The method for generating a product grabbing and placement strategy includes the following steps.

[0027] Step 101: Obtain the raw video data of the goods handling.

[0028] In this step, raw video data refers to unstructured, continuous images of the goods handling process recorded by visual sensors such as cameras. It typically includes footage of operators or robots handling goods from different angles.

[0029] Step 102: Sample the original video data to obtain a continuous video frame sequence.

[0030] In this step, sampling refers to the process of extracting key image frames from continuous raw video data according to specific time or spatial frequencies. The purpose is to convert the video stream into a discrete image sequence that can be used as input for the model. The video frame sequence refers to a set of static images (frames) obtained by sampling and arranged in chronological order. These frames continuously record the evolution of the product's state and operational actions.

[0031] Step 103: Encode the video frame sequence to generate a discrete visual vocabulary sequence.

[0032] In this step, encoding refers to the process of using a pre-trained neural network (such as a VQ-VAE encoder) to process the video frame sequence, compressing high-dimensional pixel data (such as images) and representing it as a low-dimensional, dense feature vector. Discretized visual vocabulary refers to the result obtained after encoding through further quantization operations (such as vector quantization). Each visual vocabulary is a discrete symbol or index representing a specific visual feature pattern; the entire sequence uses a string of such symbols to abstractly represent the visual evolution of the video content.

[0033] Step 104: Perform temporal dependency modeling on the visual word sequence to learn the continuous change pattern of object actions in the video content represented by the visual word sequence.

[0034] In this step, temporal dependency modeling refers to using models capable of processing sequential data (such as Transformer) to analyze the associations and dependencies between elements in a visual word sequence, thereby understanding and capturing the dynamic patterns of object motion and state changes in the video. The continuous change pattern of object motion refers to the intrinsic patterns or physical priors learned from video data through temporal dependency modeling, describing how an object (such as goods or the end effector of a robotic arm) continuously and smoothly changes from one pose or position to another.

[0035] Step 105: Based on the visual vocabulary corresponding to the current state of the goods to be operated and the visual vocabulary corresponding to the expected target state, and by utilizing the continuous change pattern of the object's actions, generate a potential action vector representing the action execution sequence required to transition from the current state to the target state.

[0036] In this step, the current state and the desired target state refer to the actual visual presentation of the goods to be operated at the start of the operation (encoded as visual vocabulary through the current frame) and the expected visual presentation when the operation is completed (encoded as visual vocabulary through the target image or description), respectively. The latent action vector refers to a low-dimensional, continuous vector representation that encodes the core information of a series of abstract action instructions required to transition from the current state to the desired target state.

[0037] Step 106: Map the potential motion vectors to the acquired real-time physical state information of the robot to generate control commands that drive the robot's robotic arm to perform goods grasping and placing operations.

[0038] In this step, the robot's real-time physical state information refers to the data describing its own motion state acquired in real time through the robot's sensors (such as encoders and force sensors), such as the angles and speeds of each joint, and the position and orientation of the end effector. Control commands refer to the specific commands that are ultimately output to the robot controller and can directly drive the motors of each joint of the robotic arm, such as target position, speed, or torque commands, to achieve the grasping or placement of goods.

[0039] In this embodiment, the continuous change patterns of object movements are automatically learned from unlabeled raw videos. These patterns are then combined with the target state to generate abstract potential action vectors, which are ultimately mapped to precise control commands that the robot can execute. This method effectively reduces the reliance on large amounts of manually labeled data, improves the model's adaptability and generalization ability to unseen scenarios (such as different goods or placement postures), and ensures the continuity and physical rationality of the generated actions through temporal modeling. This enables highly robust and efficient automated grasping and placement operations in complex dynamic environments.

[0040] In one embodiment of this specification, the original video data is sampled to obtain a continuous sequence of video frames, including:

[0041] Step 1: Decode the raw video data to obtain the raw image stream;

[0042] In this step, decoding refers to the process of processing the compressed and encoded raw video data (such as an MP4 file) using a video decoder (such as an H.264 or HEVC decoder) to restore it into a series of uncompressed or easily processed raw image frames (such as an RGB pixel array). The raw image stream refers to all the unsampled image frames obtained after decoding, arranged in chronological order. It is a continuous and usually high frame rate sequence of images that completely records the scene at every moment in the video.

[0043] Step 2: Extract image frames from the original image stream at preset time intervals to form a continuous video frame sequence.

[0044] In this step, the preset time interval refers to a pre-set time parameter used to control the sampling frequency (e.g., taking one frame every 0.1 seconds or every 5 frames). An image frame refers to a single still image extracted from the original image stream at a specific point in time, and is the basic unit that constitutes the video frame sequence.

[0045] In this embodiment, the video frame sequence of the input model is ensured to retain the temporal continuity of the actions, while the controllable downsampling reduces the subsequent computational burden and improves processing efficiency. At the same time, it avoids the problem of loss of key action information that may be caused by random sampling.

[0046] In one embodiment of this specification, encoding a video frame sequence to generate a discretized visual vocabulary sequence includes:

[0047] Step 1: Map the video frame sequence to a continuous low-dimensional space with a feature dimension lower than that of the original video frame data to obtain a continuous feature representation;

[0048] In this step, mapping refers to the operation of transforming high-dimensional raw data (image pixels) into a feature space of another dimension through specific mathematical transformations or neural networks (such as 3D convolutional neural networks). Feature dimension refers to the length or size of the data representation vector. The original video frame data typically has a high dimensionality, while the dimension of a continuous low-dimensional space is much lower, used to extract and compress core features. A continuous low-dimensional space refers to a low-dimensional mathematical space composed of real-valued vectors. In this space, each point (vector) represents a dense, continuous feature of a video frame or frame sequence. Continuous feature representation refers to the real-valued vectors obtained after mapping that represent the content of video frames in a continuous low-dimensional space. It captures the semantic and structural information of the image while maintaining numerical continuity.

[0049] Step 2: Quantize the continuous feature representation into the closest discrete codeword in the predefined codebook to generate a discretized visual vocabulary sequence.

[0050] In this step, the predefined codebook refers to a pre-trained set containing a fixed number of discrete vectors. Each vector is called a codeword, representing a typical visual feature pattern. The closest discrete codeword refers to calculating the distance (e.g., Euclidean distance) between the continuous feature vector and all codewords in the codebook during quantization, and selecting the codeword with the smallest distance as its quantization result.

[0051] In this embodiment, by compressing and quantizing high-dimensional, continuous visual data into low-dimensional, discrete symbol sequences, the abstraction and structuring of visual information are effectively achieved. This process not only significantly reduces the data dimensionality and computational complexity that subsequent temporal models need to process, but also introduces inductive bias through a predefined codebook, enhancing the model's ability to recognize and generalize similar visual patterns. Discretization also provides stable and semantically clear input units for subsequent temporal modeling (such as autoregressive prediction based on Transformer), enabling the model to learn the evolutionary patterns of object movements in video content more efficiently and accurately.

[0052] In one embodiment of this specification, temporal dependency modeling is performed on visual word sequences to learn the continuous change patterns of object actions in video content represented by the visual word sequences, including:

[0053] Step 1: Use historical visual word sequences as training samples to perform autoregressive training on the time series model; the goal of autoregressive training is to predict the visual words at the next moment based on the historical visual words at the current moment and the previous ones.

[0054] In this step, the historical visual vocabulary sequence refers to a series of discrete visual symbols (codeword indices) quantized from video frames up to a certain point in time. Arranged chronologically, they represent the evolutionary history of visual information up to that point. A temporal model refers to a machine learning model specifically designed to process sequential data, capable of capturing the dependencies between elements within a sequence. In this invention, it specifically refers to architectures such as Transformer, RNN, or LSTM, suitable for modeling long-term dependencies. Autoregressive training is a method for training a sequence generation model. Specifically, at each prediction step, the model uses all previously generated (or given) sequence elements as input to predict the next sequence element.

[0055] Step 2: Through autoregressive training, the parameters of the time series model are internalized to represent the continuous change pattern of object actions contained in the temporal evolution of visual vocabulary.

[0056] In this step, internalizing representations refers to the automatic adjustment and storage of patterns in the data by the model's internal parameters (such as weights and biases) through training. This process is implicit, enabling the model to understand and reproduce specific patterns. The continuous change pattern of object actions refers to the knowledge ultimately implicitly encoded by the parameters of the time-series model through training. This knowledge allows the model to understand and predict how the state (such as position and pose) of an object will naturally and continuously evolve to the next state within a specific visual context (historical vocabulary).

[0057] In this embodiment, an autoregressive training paradigm enables the temporal model to learn and internalize the dynamic physical laws inherent in video content from discrete visual word sequences in an unsupervised manner. It does not rely on any manually labeled action tags, greatly improving the naturalness of policy generation and its ability to generalize predictions of unseen scenes.

[0058] In one embodiment of this specification, based on the visual vocabulary corresponding to the current state of the goods to be operated and the visual vocabulary corresponding to the desired target state, and utilizing the continuous change pattern of object actions, a potential action vector representing the action execution sequence required to transition from the current state to the target state is generated, including:

[0059] Step 1: Based on the visual vocabulary corresponding to the current state and the visual vocabulary corresponding to the desired target state, generate forward prediction trajectory and backward prediction trajectory respectively through a time series model;

[0060] In this step, the visual vocabulary corresponding to the current state refers to the discrete symbols obtained by encoding and quantizing the image of the goods to be operated at the start of the operation (i.e., the current state). The visual vocabulary corresponding to the desired target state refers to the discrete symbols obtained by encoding and quantizing the image (or descriptive image) of the goods to be operated at the desired completion time of the operation (i.e., the target state). The forward prediction trajectory refers to a series of future visual vocabulary sequences obtained by using a temporal model to predict multiple steps forward (i.e., towards the future time direction) using the visual vocabulary corresponding to the current state as the starting point. The backward prediction trajectory refers to a series of historical visual vocabulary sequences obtained by using a temporal model to predict multiple steps backward (i.e., towards the past time direction) using the backward autoregressive prediction trajectory corresponding to the desired target state as the starting point.

[0061] Step 2: Align and fuse the forward and backward predicted trajectories in the latent action space to obtain the initial latent action sequence;

[0062] In this step, the potential action space refers to an abstract low-dimensional continuous vector space, where each point (vector) is used to encode and represent a set of potential, robot-independent abstract action instructions.

[0063] Step 3: In the optimization process of the potential action sequence, environmental constraints are introduced for iterative optimization to obtain the optimized potential action sequence;

[0064] In this step, environmental constraints refer to the physical rules or task requirements that must be followed when performing actions, such as the kinematic limitations of the robotic arm, avoiding collisions with obstacles, and the stability requirements of the goods. Iterative optimization refers to the process of starting with an initial potential sequence of actions and repeatedly (iterically) adjusting and improving it according to the environmental constraints and optimization objectives to seek a better solution.

[0065] Step 4: Based on the optimized potential action sequence, multiple candidate sequences are selected using preset execution success rate and accuracy indicators;

[0066] In this step, execution success rate and accuracy metrics refer to quantitative standards used to evaluate the quality of an action sequence. Execution success rate refers to the probability that the sequence will successfully complete the grasping or placement task in simulation or historical experience. Accuracy metrics may refer to the deviation between the final placement position and the target position, the error in the object's attitude angle, etc.

[0067] Step 5: Merge the selected candidate sequences to generate potential action vectors for final mapping.

[0068] In this step, fusion refers to combining information from multiple selected candidate sequences (such as their action vectors), for example, by averaging, weighted averaging, or selecting the most consistent parts.

[0069] In this embodiment, by combining the bidirectional programming ideas of forward and backward prediction, and introducing iterative optimization and multi-candidate sequence fusion mechanisms under environmental constraints, the rationality, security and robustness of the generation strategy are significantly improved.

[0070] In one embodiment of this specification, based on the visual vocabulary corresponding to the current state and the visual vocabulary corresponding to the desired target state, a forward prediction trajectory and a backward prediction trajectory are generated respectively through a temporal model, including:

[0071] Step 1: Using the visual words corresponding to the current state as the initial input, predict the visual words for the next multiple steps through autoregression using a temporal model, and connect the latent state vectors output at each step in sequence to form a positive prediction trajectory.

[0072] In this step, autoregressive prediction refers to a sequence generation method where the model uses the output of the previous step as part of the input for the next step in each prediction, thus progressively generating a complete sequence. Here, it refers to predicting future words starting from the current word. The latent state vector refers to the intermediate vector representation generated and transmitted by the internal network layers of a temporal model (such as the Decoder layer of a Transformer) at each prediction step, representing the current context and historical information. It encodes the semantic and state information of the sequence up to the current step.

[0073] Step 2: Using the visual words corresponding to the desired target state as the initial input, predict the visual words of multiple historical steps through inverse autoregression using a time series model. Connect the hidden state vectors output at each step in sequence to form the inverse prediction trajectory.

[0074] In this step, inverse autoregressive prediction refers to a special prediction method whose process is the opposite of autoregressive prediction. It starts with the desired target state (considered the end point of the sequence) and predicts the historical steps before reaching the target in reverse, generating a backtracking sequence.

[0075] In this embodiment, the generated motion trajectory is ensured to reflect the dynamic transition from the initial state to the target state more accurately and smoothly, thereby improving the coherence and physical rationality of the strategy.

[0076] In one embodiment of this specification, the forward and backward predicted trajectories are aligned and fused in the potential action space to obtain an initial potential action sequence, including:

[0077] Step 1: Map the forward and backward predicted trajectories to the same potential action space;

[0078] In this step, mapping to the same latent action space means transforming each latent state vector in the forward and backward predicted trajectories into a vector representation in the same low-dimensional continuous space (i.e., the latent action space) through a linear transformation or neural network projection layer. This allows for comparison and manipulation within a unified mathematical space.

[0079] Step 2: Based on the temporal correspondence between the hidden state vectors in the forward predicted trajectory and the hidden state vectors in the reverse predicted trajectory, perform point-to-point matching on the hidden state vectors in the forward predicted trajectory and the reverse predicted trajectory to obtain the matching results.

[0080] In this step, the temporal correspondence refers to the logical matching relationship between points on the timeline of the two trajectories. Typically, the starting point of the forward trajectory (representing the current state) and the ending point of the reverse trajectory (representing the last step before reaching the goal) should be aligned at the same physical moment; the ending point of the forward trajectory and the starting point of the reverse trajectory (representing the target state) should be aligned at the same physical moment. Intermediate points are interpolated or matched according to the time ratio. Point-to-point matching refers to finding the corresponding (representing the same physical moment in time) hidden state vector in the reverse trajectory for each hidden state vector in the forward trajectory, based on the above temporal correspondence, forming a one-to-one vector pair.

[0081] Step 3: Based on the matching results, calculate the similarity weight between each hidden state vector in the forward predicted trajectory and the corresponding hidden state vector in the reverse predicted trajectory.

[0082] In this step, the similarity weight is a calculated value used to quantify the degree of similarity or consistency between a vector in the forward trajectory and its corresponding vector in the reverse trajectory. The higher the similarity, the greater the weight, indicating that the pair should account for a larger proportion in the final fusion.

[0083] Step 4: Based on similarity weights, the corresponding latent state vectors in the forward and reverse predicted trajectories are weighted and fused to generate an initial potential action sequence.

[0084] In this embodiment, it is possible to effectively reconcile any contradictions or uncertainties that may exist between two trajectories (for example, the forward-predicted action may not reach the target precisely, and the reverse-tracking path may not meet the physical starting conditions), generating an initial action sequence that is smoother and more consistent in the action space, while simultaneously satisfying the start and end point constraints. This fusion mechanism improves the overall coherence and logical consistency of the generation strategy.

[0085] In one embodiment of this specification, during the optimization of a potential action sequence, environmental constraints are introduced for iterative optimization, including:

[0086] Step 1: Construct a simulation environment; the simulation environment includes obstacle models and physical properties of goods based on real-time perception reconstruction.

[0087] In this step, building a simulation environment refers to using computer software (such as a physics simulation engine, like MuJoCo, PyBullet, or NVIDIA Isaac Sim) to create a virtual operating scene that can simulate real-world physical laws (such as gravity, collisions, and friction). Real-time perception and reconstruction refers to processing and analyzing data collected in real time by robot sensors (such as depth cameras and LiDAR) in the real environment, and creating a corresponding virtual model with geometric shape and physical properties in the simulation environment. Obstacle models refer to virtual 3D geometric models in the simulation environment used to represent objects (such as shelves or other goods) that hinder the movement of the robotic arm, and assigning them collision properties. Goods physical properties refer to the physical parameters set for the virtual goods model to be operated in the simulation environment, including but not limited to mass, center of mass, moment of inertia, coefficient of friction, and elastic coefficient, which determine its dynamic behavior in the simulation.

[0088] Step 2: Decode the potential action sequence of the current iteration into the simulated motion trajectory of the robotic arm, and simulate its execution in the simulation environment to obtain the simulation execution results;

[0089] In this step, simulation execution refers to driving the virtual robotic arm model to move according to the simulated motion trajectory in the constructed simulation environment, and running the physics engine to calculate its interaction with the virtual environment (obstacles, goods). Simulation execution results refer to a series of output data obtained after the simulation execution is completed, including but not limited to: whether the task was successfully completed (e.g., successful grabbing and placement), the final state of the goods, whether a collision occurred, the joint torque of the robotic arm, energy consumption, etc.

[0090] Step 3: Based on the simulation results, determine the optimization objectives; the optimization objectives should at least include safety constraints and task performance indicators.

[0091] In this step, safety constraints refer to the hard conditions that must be met and are an important component of the optimization objective. These typically include: no collisions (with obstacles or the object itself), joint angles / velocities / torques not exceeding limits, and the object remaining stable and not falling during the grasping process. Violating these constraints usually leads to a deterioration of the optimization objective or task failure. Task performance metrics refer to the quantitative standards for evaluating the quality of task completion and are another important component of the optimization objective. Examples include: placement accuracy, operation completion time, motion smoothness, and energy consumption.

[0092] Step 4: Based on the optimization objective, update the potential action sequence using the policy gradient method and perform iterative optimization until the preset convergence condition is met.

[0093] In this step, the preset convergence criteria refer to the standards used to determine when the optimization process can terminate. Common criteria include: performance improvement being less than a certain threshold after multiple consecutive iterations, reaching the maximum number of iterations, or the overall optimization target value reaching a preset standard.

[0094] In this embodiment, by pre-executing and evaluating potential action sequences in a virtual environment, problems that are difficult to detect at the purely visual / planning level can be proactively identified and corrected, such as interference with environmental obstacles, violations of robot kinematics / dynamics limits, or actions that cause instability in goods. Through the policy gradient method, the system can automatically and efficiently search for optimal solutions that satisfy complex multi-objectives (safety and task performance).

[0095] In one embodiment of this specification, the potential action sequence is updated using a policy gradient method based on an optimization objective, including:

[0096] Step 1: Represent the potential action sequence as a probability distribution;

[0097] In this step, the probability distribution refers to using a statistical model (such as a multivariate Gaussian distribution) to characterize the potential sequence of actions. The parameters of this distribution (such as the mean vector and covariance matrix) define the most likely sequence form and its possible range of variation, transforming the sequence from a deterministic value into a random variable that can describe uncertainty and be sampled.

[0098] Step 2: In each optimization iteration, sample multiple candidate potential action sequences from the probability distribution;

[0099] Step 3: Perform simulation evaluation on each candidate potential action sequence to obtain the corresponding optimization target value;

[0100] In this step, simulation evaluation refers to executing the robotic arm motion corresponding to each candidate sequence in a physical simulation environment and simulating its interaction with the environment. The optimization target value refers to a scalar value calculated after simulation evaluation based on a predefined comprehensive evaluation function (including task performance and safety indicators), quantifying the merits of the candidate sequence. A higher value generally indicates a better sequence.

[0101] Step 4: Calculate the policy gradient based on the optimization objective value, and update the parameters of the probability distribution using the policy gradient;

[0102] In this step, the policy gradient refers to the derivative (gradient) of the objective value with respect to the parameters of the probability distribution. It indicates how the distribution parameters should be adjusted in direction and magnitude to improve the expected performance of the sequence sampled from the distribution. Updating the parameters of the probability distribution means using the calculated policy gradient to adjust parameters such as the mean and covariance of the probability distribution through an optimization algorithm (such as gradient ascent), so that the distribution moves towards a region that produces a higher objective value.

[0103] Step 5: After updating the parameters of the probability distribution, monitor whether the preset stability index in the simulation evaluation results meets the preset safety threshold.

[0104] In this step, the simulation evaluation results refer to the detailed data collected during the simulation evaluation process. This includes not only the final optimization target value but also intermediate process data used to calculate the stability indices. Preset stability indices are quantitative standards used to measure the physical safety and robustness of the action, such as the maximum acceleration at the robotic arm's end effector, peak joint torque, cargo slippage, or collision detection markers. Preset safety thresholds are the upper or lower limits allowed by the stability indices, serving as boundaries for determining whether the action is safe and feasible.

[0105] Step 6: If the preset stability index does not meet the safety threshold, the update process will be backtracked and the update step size will be reduced.

[0106] In this step, backtracking refers to the process where, when the stability index is detected to exceed the safety threshold, it is determined that the parameter update may have introduced insecurity. Therefore, the update is revoked, and the distribution parameters are rolled back to their state before the update.

[0107] Step 7: When the preset convergence condition is met, terminate the optimization and output the final potential action sequence.

[0108] In this step, convergence criteria refer to the criteria used to determine when the optimization process terminates, such as performance improvement being less than a threshold, reaching the maximum number of iterations, or the optimization target value reaching a satisfactory level.

[0109] In this embodiment, by modeling potential action sequences as probability distributions and combining them with policy gradient optimization, the system improves task performance while introducing a real-time security monitoring and backtracking mechanism to ensure that the generated action sequences are both efficient and reliable and strictly meet physical security constraints, thereby enhancing the robustness of the policy and the feasibility of actual deployment.

[0110] In some other embodiments of this specification, image frames are extracted from the original image stream at preset time intervals to form a continuous video frame sequence, including:

[0111] Candidate frames are extracted according to a fixed beat and their timestamps are recorded.

[0112] Perform a perceptual hash operation on adjacent candidate frames. If the hash Hamming distance is less than a set threshold, the two frames are determined to be identical, and the latter frame is discarded, while its timestamp is incorporated into the valid window of the former frame.

[0113] A frame is included in the sequence only if the duration of the effective window is not less than the preset lower limit and the content of the candidate frame has changed relative to the previous retained frame.

[0114] The frames are output in the order of their timestamps to form a temporally continuous and non-redundant video frame sequence.

[0115] In some other embodiments of this specification, the video frame sequence is mapped to a continuous low-dimensional space with a feature dimension lower than that of the original video frame data to obtain a continuous feature representation, including:

[0116] The first feature vector is obtained by performing forward computation on each frame of the image through a convolutional encoder. The number of elements in the first feature vector is less than the number of pixels in a single frame.

[0117] The first feature vector is input into a two-layer fully connected network to obtain the second and third feature vectors, respectively. The third feature vector has fewer elements than the second feature vector, and the second feature vector has fewer elements than the first feature vector.

[0118] Perform L2 normalization on the third eigenvector to make the vector magnitude 1, and obtain the normalized eigenvector;

[0119] The normalized feature vectors of adjacent frames are arranged into a sequence in chronological order, and a moving average is performed on each vector in the sequence with its preceding vector to obtain a continuous feature sequence with the same length as the original frame sequence, which is used as the continuous feature representation.

[0120] In this embodiment, the convolutional encoder refers to a feature extraction network composed of convolutional layers and pooling layers. It takes an image as input and outputs a vector of fixed length with fewer elements than the number of pixels. The fully connected network refers to a feedforward network where each neuron in one layer is connected to all neurons in the layer above. This network compresses the first feature vector step-by-step into a second feature vector with fewer elements, and then further into a third feature vector with even fewer elements. The moving average method uses a weighted sum of the normalized feature vector of the current frame and the normalized feature vector of the previous frame to suppress single-frame jitter and maintain temporal smoothness. This embodiment transforms the original image sequence into a unified feature sequence with far fewer elements than pixels and continuous temporal sequence, preserving key visual information while reducing the computational and storage burden of subsequent temporal modeling.

[0121] Based on the same general inventive concept, this invention also protects a system for generating a goods picking and placing strategy, such as... Figure 2 As shown, Figure 2 This is a schematic diagram of the structure of the product grabbing and placement strategy generation system provided in an embodiment of the present invention. The product grabbing and placement strategy generation system provided by the present invention will be described below. The product grabbing and placement strategy generation system described below can be referred to in correspondence with the product grabbing and placement strategy generation method described above.

[0122] The system for generating product picking and placement strategies includes:

[0123] The data acquisition module 201 is used to acquire raw video data of goods handling;

[0124] The data sampling module 202 is used to sample and process the raw video data to obtain a continuous video frame sequence;

[0125] The visual vocabulary module 203 is used to encode the video frame sequence to generate a discrete visual vocabulary sequence;

[0126] The temporal dependency module 204 is used to perform temporal dependency modeling on the visual word sequence in order to learn the continuous change pattern of object actions in the video content represented by the visual word sequence.

[0127] The latent action module 205 is used to generate a latent action vector that represents the action execution sequence required to transition from the current state to the target state, based on the visual vocabulary corresponding to the current state of the goods to be operated and the visual vocabulary corresponding to the expected target state, and by utilizing the continuous change law of the object's actions.

[0128] The instruction generation module 206 is used to map potential motion vectors with the acquired real-time physical state information of the robot to generate control instructions that drive the robotic arm to perform goods grasping or placement operations.

[0129] Figure 3 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention.

[0130] like Figure 3 As shown, the electronic device may include a processor 310, a communications interface 320, a memory 330, and a communication bus 340. The processor 310, communications interface 320, and memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions stored in the memory 330 to execute a method for generating a goods picking and placing strategy.

[0131] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0132] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0133] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating a goods picking and placing strategy, characterized in that, include: Obtain raw video data of goods handling; The original video data is sampled to obtain a continuous sequence of video frames; The video frame sequence is encoded to generate a discrete visual vocabulary sequence; Temporal dependency modeling is performed on the visual word sequence to learn the continuous change pattern of object actions in the video content represented by the visual word sequence. Using the visual words corresponding to the current state as the initial input, the visual words for the next multiple steps are predicted autoregressively through a temporal model. The latent state vectors output at each step are connected in sequence to form a positive prediction trajectory. Using the visual words corresponding to the desired target state as the initial input, the time series model is used to predict the visual words of multiple historical steps in an inverse autoregressive manner. The hidden state vectors output at each step are connected in sequence to form an inverse prediction trajectory. Map the forward predicted trajectory and the reverse predicted trajectory to the same potential action space; Based on the temporal correspondence between the latent state vectors in the forward and backward predicted trajectories, point-to-point matching is performed on the latent state vectors in the forward and backward predicted trajectories to obtain the matching results. Based on the matching results, calculate the similarity weight between each hidden state vector in the forward predicted trajectory and the corresponding hidden state vector in the reverse predicted trajectory. Based on similarity weights, the corresponding latent state vectors in the forward and reverse predicted trajectories are weighted and fused to generate an initial potential action sequence. Construct a simulation environment; wherein, the simulation environment includes an obstacle model and the physical properties of the goods based on real-time perception reconstruction; The potential action sequence of the current iteration is decoded into the simulated motion trajectory of the robotic arm, and then simulated and executed in the simulation environment to obtain the simulation execution result; Based on the simulation results, optimization objectives are determined; the optimization objectives include at least security constraints and task performance indicators. The potential action sequence is represented as a probability distribution; In each optimization iteration, multiple candidate potential action sequences are sampled from the probability distribution; Each candidate potential action sequence is simulated and evaluated to obtain the corresponding optimization target value; The policy gradient is calculated based on the optimization objective value, and the parameters of the probability distribution are updated using the policy gradient. After updating the parameters of the probability distribution, monitor whether the preset stability index in the simulation evaluation results meets the preset safety threshold. If the preset stability index does not meet the safety threshold, the update process is backtracked and the update step size is reduced. When the preset convergence condition is met, the optimization is terminated and the final potential action sequence is output. Based on the final potential action sequence, multiple candidate sequences are selected using preset execution success rate and accuracy indicators; The selected candidate sequences are fused to generate potential action vectors for final mapping; The potential motion vectors are mapped to the acquired real-time physical state information of the robot to generate control commands that drive the robot's robotic arm to perform goods grasping and placing operations.

2. The method for generating a goods picking and placing strategy according to claim 1, characterized in that, The step of sampling the original video data to obtain a continuous sequence of video frames includes: The original video data is decoded to obtain the original image stream; Image frames are extracted from the original image stream at preset time intervals to form a continuous video frame sequence.

3. The method for generating a goods picking and placing strategy according to claim 1, characterized in that, The process of encoding the video frame sequence to generate a discrete visual vocabulary sequence includes: The video frame sequence is mapped to a continuous low-dimensional space with a feature dimension lower than that of the original video frame data to obtain a continuous feature representation; The continuous feature representation is quantized into the closest discrete codeword in a predefined codebook to generate a discretized visual vocabulary sequence.

4. The method for generating a goods picking and placing strategy according to claim 1, characterized in that, The step of performing temporal dependency modeling on the visual word sequence to learn the continuous change pattern of object actions in the video content represented by the visual word sequence includes: The time series model is trained using historical visual word sequences as training samples; the goal of the autoregressive training is to predict the visual words at the next moment based on the historical visual words at the current moment and the previous ones. Through the autoregressive training, the parameters of the time series model are internalized to represent the continuous change pattern of object actions contained in the temporal evolution of visual vocabulary.

Images