A contact cone based model robot guidance control method and system

Abstract

The application discloses a model robot guidance control method and system based on a contact cone, which comprises the following steps: acquiring contact information, modeling contact cone geometric constraints based on the contact information, and dividing a gripping safety zone; geometrically discretely sampling the contact cone, acquiring a sampling direction set, mapping the sampling direction set into an embedding vector, splicing the embedding vector, and acquiring a contact cone geometric token; fusing multi-modal information and the contact cone geometric token, inputting the fusion result into a preset architecture, and generating an action sequence; updating the contact information in real time, re-modeling the contact cone geometric constraints according to the updated contact information, and generating a new round of contact cone geometric tokens. The application realizes the generation and stable control of the physical feasibility guidance of the robot by structurally coding the contact cone geometric constraints in contact mechanics through discrete direction sampling and introducing VLA.

Description

Technical Field

[0001] This invention relates to the field of robot manipulation and control, and specifically to a model robot guidance and control method and system based on a contact cone. Background Technology

[0002] Visual-language-action (VLA) models, as the mainstream research paradigm for embodied robot control, have made significant progress. They can generate robot action sequences based on natural language commands, greatly improving the robot's task generalization ability and the convenience of human-robot interaction. Meanwhile, contact cones, as the core geometric model describing the force constraints of robot-object contact, have been widely used in physical constraint modeling for robot grasping, manipulation, and other scenarios.

[0003] To further improve robot performance in physical interaction scenarios, existing VLA methods are gradually incorporating force / contact information to enhance the rationality of motion generation. Meanwhile, the contact cone (also known as friction cone) theory from traditional contact mechanics has been applied in low-level robot optimization control, such as in model predictive control (MPC) and whole-body control scenarios. By modeling the constraints of contact forces, it assists the low-level controller in achieving more stable motion execution.

[0004] Despite this, existing technologies still have significant shortcomings, making it difficult to meet the demands of robotic tasks in complex, multi-contact scenarios. On one hand, current VLA methods lack explicit geometric modeling of the feasible domain of contact force directions. They can only initially integrate force magnitude, torque feedback, or simple contact detection information, failing to provide a systematic and structured description and constraint of the "feasible geometric space of force directions." This leads to generated actions easily exceeding friction conditions, resulting in problems such as slippage, instability, and even falls. On the other hand, while traditional contact cone theory can achieve low-level control constraints, it has not been transformed into structured inputs that can be directly processed by high-level VLA models. Furthermore, it has not established a direct physical feasibility guidance mechanism in the action generation stage, causing a disconnect between high-level action planning and low-level physical constraints, making it difficult to fully leverage the supporting role of contact mechanics theory. Summary of the Invention

[0005] Based on this, the purpose of this invention is to provide a model robot guidance and control method and system based on contact cones, which aims to solve the problem that the contact cones are not converted into structured inputs that can be processed by VLA high-level models, resulting in a lack of physically feasible guidance mechanisms and preventive physical constraints.

[0006] To achieve the above objectives, this invention proposes a model robot guidance and control method based on a contact cone, the method comprising: Obtain contact information, perform contact cone geometric constraint modeling based on the contact information, and divide the gripping safety zone; Geometric discretization sampling is performed on the contact cone to obtain a set of sampling directions. The set of sampling directions is mapped to an embedding vector. The embedding vectors are concatenated to obtain a contact cone geometric token. By fusing multimodal information and the contact cone geometric token, the fusion result is input into a preset architecture to generate an action sequence; The contact information is updated in real time. Based on the updated contact information, the contact cone geometric constraints are remodeled, and a new round of contact cone geometric tokens is generated.

[0007] According to one aspect of the above technical solution, in the steps of acquiring contact information, performing contact cone geometric constraint modeling based on the contact information, and dividing the gripping safety zone: Real-time acquisition of robot end-effector contact information, the contact information including at least the contact position. Contact normal vector and coefficient of friction ; Based on the contact information, a contact cone geometric model is constructed:

[0008] in, The contact normal vector, This is the contact force vector.

[0009] According to one aspect of the above technical solution, in the step of geometrically discretizing the contact cone to obtain the set of sampling directions: The sampling number of the contact cone geometric model is determined, the contact cone geometric model is divided based on the sampling number, the polar angle is determined simultaneously, and the local sampling direction is generated; The local sampling direction is transformed to world coordinates using a preset rotation matrix to obtain a set of sampling directions composed of several transformed local sampling directions.

[0010] According to one aspect of the above technical solution, the steps after obtaining the sampling direction set are as follows: Any sampling direction in the normalized set of sampling directions is mapped to an embedding vector of fixed dimension, wherein the embedding vector contains position encoding; A preset number of embedding vectors are concatenated, and a single independent geometric token is obtained through linear projection. The preset number is equal to the number of samples. The single independent geometric token is an independent geometric token under a single contact point. Several independent geometric tokens under a single contact point form a geometric token sequence with multiple contact points.

[0011] According to one aspect of the above technical solution, the step of fusing multimodal information and the contact cone geometric token, inputting the fusion result into a preset architecture, and generating an action sequence is as follows: The system receives visual information encoded by a visual encoder, instruction information encoded by a language encoder, and a geometric token sequence, respectively, and fuses them to generate a unified input sequence. The unified input sequence is then input into a preset VLA model structure to obtain an autoregressive action sequence.

[0012] According to one aspect of the above technical solution, based on the candidate actions in the autoregressive generated action sequence, the robot end-effector posture at the next moment is predicted by the forward dynamics module, the contact point velocity and expected contact force are calculated using the predicted robot end-effector posture, and the expected direction is obtained by normalization. Calculate the matching degree between the expected direction and the set of sampling directions:

[0013] in, As expected, Let be the set of sampling directions, if , If the tolerance threshold is met, then within the geometric space, the current candidate action is retained for robot execution. If so, then explicit exclusion will be performed.

[0014] The present invention also provides a contact cone-based model robot guidance and control system, wherein the contact cone-based model robot guidance and control system is used in the above-described contact cone-based model robot guidance and control method, and the system includes: The constraint modeling module is used to acquire contact information, perform contact cone geometric constraint modeling based on the contact information, and divide the gripping safety zone. The token acquisition module is used to perform geometric discrete sampling on the contact cone, obtain a set of sampling directions, map the set of sampling directions to an embedding vector, and concatenate the embedding vectors to obtain a contact cone geometric token. An action generation module is used to fuse multimodal information and the contact cone geometric token, input the fusion result into a preset architecture, and generate an action sequence. The dynamic update module is used to update the contact information in real time, remodel the contact cone geometric constraints based on the updated contact information, and generate a new round of contact cone geometric tokens.

[0015] The present invention also proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the contact cone-based model robot guidance and control method described above.

[0016] The present invention also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the contact cone-based model robot guidance and control method described above.

[0017] In summary, the contact cone-based model robot guidance and control method provided by this invention proposes a contact cone geometry discrete sampling and encoding mechanism. This mechanism discretizes and structures the feasible domain of the contact cone direction into independent geometric tokens, transforming the mechanical geometric space into a structured input that can be processed by the higher-level VLA (Virtual Aspect-Based Logic) layer. A direct correlation is established between the contact cone geometric constraints and the action output in the VLA action generation stage, achieving a hybrid guidance of explicit verification and implicit modulation, which differs from existing force feedback fusion or low-level optimization methods. This invention achieves action generation and stable control for physically feasible robot guidance by structurally encoding the contact cone geometric constraints in contact mechanics through discrete direction sampling and introducing them into VLA.

[0018] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0019] Figure 1 This is a flowchart of the contact cone-based model robot guidance and control method in Embodiment 1 of the present invention; Figure 2 This is a flowchart of the contact cone-based model robot guidance and control method in Embodiment 2 of the present invention; Figure 3 This is a schematic diagram of the structure of the contact cone-based model robot guidance and control system in Embodiment 3 of the present invention; Figure 4 This is a structural block diagram of the electronic device in Embodiment 5 of the present invention. Detailed Implementation

[0020] To make the objectives, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Several embodiments of the present invention are shown in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that the disclosure of the present invention will be more thorough and complete.

[0021] It should be noted that when an element is referred to as being "fixed to" another element, it can be directly on the other element or there may be an intervening element. When an element is considered to be "connected" to another element, it can be directly connected to the other element or there may be an intervening element. The terms "vertical," "horizontal," "left," "right," "upper," "lower," and similar expressions used herein are for illustrative purposes only and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as limiting the invention.

[0022] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances. The term "and / or" as used herein includes any and all combinations of one or more of the related listed items.

[0023] Example 1 like Figure 1 The diagram shows a flowchart of a contact cone-based model robot guidance and control method according to Embodiment 1 of the present invention. The contact cone-based model robot guidance and control method includes the following steps S11-S14, wherein: S11. Obtain contact information, perform contact cone geometric constraint modeling based on the contact information, and divide the gripping safety zone.

[0024] In this embodiment, contact information is acquired in real time from the robot's end effector force / torque sensor or vision-tactile fusion sensing module. The robot's end effector can be its arm or leg, and contact information is acquired through sensors mounted on it. The contact location is the point where the robot's end effector touches the object. Based on the acquired information, a contact cone geometric constraint model is created. This model represents a cone-shaped feasible region, i.e., the safe area for the robot to grasp the object, and the direction of the robot's applied force is within this safe area.

[0025] S12. Perform geometric discrete sampling on the contact cone to obtain a set of sampling directions, map the set of sampling directions to an embedding vector, and concatenate the embedding vectors to obtain a contact cone geometric token.

[0026] After completing the contact cone geometric constraint modeling, the range within the contact cone geometric constraint model is discretized and sampled. First, the contact cone geometric constraint model region needs to be divided, and the edge of the contact cone geometric constraint model is split into a preset number of standard directions.

[0027] After determining the number of samples, a preset number of standard force directions are separated from the edge of the contact cone geometric constraint model to obtain a set of sampling directions. The set of sampling directions is normalized and mapped to obtain a fixed-dimensional embedding vector. The embedding vectors are then concatenated to obtain the contact cone geometric token.

[0028] S13. Fuse the multimodal information and the contact cone geometric token, input the fusion result into the preset architecture, and generate an action sequence.

[0029] The robot fuses and splices the multimodal information of the external environment acquired in real time with the contact cone geometric tokens acquired in the previous step, and then inputs the spliced ​​unified input sequence into the preset VLA model to generate an action sequence. This sequence is finally mapped into motor commands through the low-level controller.

[0030] S14. Update the contact information in real time, remodel the contact cone geometric constraints based on the updated contact information, and generate a new round of contact cone geometric tokens.

[0031] After the action is executed, the contact information (sensor feedback) is updated in real time, the above steps are re-executed, a new geometric token is generated, and closed-loop control is achieved.

[0032] In summary, the contact cone-based model robot guidance and control method provided by this invention proposes a contact cone geometry discrete sampling and encoding mechanism. This mechanism discretizes and structures the feasible domain of the contact cone direction into independent geometric tokens, transforming the mechanical geometric space into a structured input that can be processed by the higher-level VLA (Virtual Aspect-Based Logic) layer. A direct correlation is established between the contact cone geometric constraints and the action output in the VLA action generation stage, achieving a hybrid guidance of explicit verification and implicit modulation, which differs from existing force feedback fusion or low-level optimization methods. This invention achieves action generation and stable control for physically feasible robot guidance by structurally encoding the contact cone geometric constraints in contact mechanics through discrete direction sampling and introducing them into VLA.

[0033] Example 2 Please see Figure 2 The figure shows a contact cone-based model robot guidance and control method in the second embodiment of the present invention, including steps S21-S25.

[0034] S21. Obtain contact information, perform contact cone geometric constraint modeling based on the contact information, and divide the gripping safety zone.

[0035] In this embodiment, contact information, including the contact location, is obtained in real time from the robot's end effector force / torque sensor or vision-touch fusion sensing module. Contact normal vector (Unit vector), coefficient of friction (Typical range 0.1~1.0, based on material presets or online estimation) `<vector>` is a common standard notation in linear algebra and robotics, representing a vector space consisting of ordered triples of all three-dimensional real numbers. The robot's end effector can be an arm or leg, which acquires contact information via sensors. The contact point is the point where the robot's end effector touches the object. Based on the information obtained, contact cone geometric constraints are modeled:

[0036] in, The contact normal vector, The contact force vector, For the tangential component. This model defines a... axis, semi-cone angle The cone-shaped feasible region is the safe area for the robot to grasp the object, and the direction of the robot's force application is within the safe area.

[0037] S22. Perform geometric discrete sampling on the contact cone to obtain a set of sampling directions, map the set of sampling directions to an embedding vector, and concatenate the embedding vectors to obtain a contact cone geometric token.

[0038] S221. Determine the number of samples and the direction of discrete sampling.

[0039] The contact cone geometric constraint model edge is split into One standard direction, among which The value range is 8-32, preferably... Experiments have shown that when At that time, the geometric accuracy of the contact cone geometric constraint model was insufficient, with an angle error >15°. At that time, excessively high embedding dimensions lead to a more than 30% increase in the computational cost of the VLA model. At the same time, it achieves the optimal balance between accuracy and embedding efficiency, and can cover more than 90% of the practically feasible force directions.

[0040] In the robot's local coordinate system (with) Uniform sampling is performed at the boundary (for the z-axis) to obtain the azimuth angle. Evenly divided into Equal parts: , polar angle Fixed as the boundary angle of the conical surface .

[0041] Generate local sampling direction (Unit vector):

[0042] By rotation matrix (Depend on Construction, making Transform to world coordinates:

[0043] Obtain the set of sampling directions Additionally, to enable the robot to better support or grasp objects, a higher collection density can be applied to the support direction.

[0044] S222, mapped to an embedding vector, forming a contact cone geometric token.

[0045] For the set of sampling directions Perform normalization to obtain , , For each sample direction set, the result is normalized. Mapped to a fixed-dimensional embedding vector via MLP (D=512, consistent with the VLA embedding dimension).

[0046] It should be noted that the MLP structure consists of three layers: a fully connected layer, a ReLU layer, and a LayerNorm layer. The learnable parameters are jointly optimized during VLA training.

[0047]

[0048]

[0049] Let j be the unit vector of the sampling direction of the normalized contact cone; , , These are the weight matrices for the first, second, and third layers, respectively. , , These are the bias vectors for the first, second, and third layers, respectively. , These are the output feature vectors of the first and second hidden layers, respectively; This is the final generated geometric embedding vector for the j-th sampling direction; To modify the activation function of the linear unit, it is defined as follows: This is used to introduce nonlinear expressive capabilities into networks; The layer normalization operation is used to accelerate model training and stabilize gradient propagation; D is the embedding vector dimension. In this invention, D is fixed at 512 to be consistent with the output dimension of the visual encoder and language encoder of the VLA model, ensuring that multimodal features can be directly spliced ​​and fused.

[0050] In the embedding vector Alternatively, position encoding can be added (treating direction as a sequence):

[0051] in This is a standard sinusoidal positional encoding.

[0052] After obtaining the fixed-dimensional embedding vectors, the k embedding vectors are concatenated and then linearly projected to obtain a single independent geometric token. Alternatively, the average value can be projected to form an independent geometric token for a single contact point. Several independent geometric tokens for single contact points together form a contact cone geometric token for multiple contact points. (C represents the number of contact points). Additionally, geometric semantic tags (one-hot vectors, support / anti-slip / slip) can be appended to the geometric tokens and concatenated to... .

[0053] S23. Fuse the multimodal information and the contact cone geometric token, input the fusion result into the preset architecture, and generate an action sequence.

[0054] The robot fuses and splices the multimodal information of the external environment acquired in real time with the contact cone geometric tokens acquired in the previous step, and then inputs the spliced ​​unified input sequence into the preset VLA model to generate an action sequence. This sequence is finally mapped into motor commands through the low-level controller.

[0055] The VLA model structure in this embodiment is illustrated using the decoder-only architecture of Transformer (similar to RT-X / PaLM-E) as an example.

[0056] The system receives visual information encoded by a visual encoder, instruction information encoded by a language encoder, and a geometric token sequence, respectively, and fuses them to generate a unified input sequence. This unified input sequence is then input into a preset VLA model structure to obtain an autoregressive action sequence. Specifically: Visual encoder: ViT-B / 16, which encodes RGB images into N patch embeddings. , ; Language encoder: A text embedder based on T5 / BERT that encodes instructions into M token embeddings. , ; Geometric encoder: The above contact cone geometric token sequence .

[0057] Fusion method: All features have been unified to dimension D and concatenated into a unified input sequence. :

[0058] (P=k or Add learnable positional encoding and then input Transformer.

[0059] The fused unified input sequence is input into the VLA model structure in this embodiment, and an autoregressive action sequence is generated. Each of them It is either a discrete motion token (vocabulary size 4096) or a continuous motion vector (joint velocity / endpoint pose increment, dimension equal to robot degrees of freedom).

[0060] S24. Provide physical guidance for the robot's movements and add physical rule constraints.

[0061] S241. Calculate the expected contact force direction of the candidate action.

[0062] For action a (joint velocity) in each action sequence or terminal velocity ), calculated via the forward dynamics module (real-time simplified simulator or pre-trained neurodynamic model): Predict the pose / velocity of the robot's end effector at the next moment based on the current joint state q and the candidate action a; The candidate motion is mapped to the contact point velocity using the Jacobian matrix J. ; Calculate the expected contact force using the contact stiffness model :

[0063] in, : Expected contact force vector (unit: Newton, N), representing the predicted value of the contact force that will be generated between the robot end effector and the object after the candidate action is performed; Normal contact stiffness coefficient (unit: Newton / meter, N / m) represents the ability of the contact surface to resist normal deformation. Its value is pre-calibrated or estimated online based on the characteristics of the contact material. : The normal position coordinates of the current contact point (unit: meters, m), that is, the projection value of the contact point in the direction of the contact normal vector n; : The initial contact point normal position coordinates (unit: meters, m) when there is no deformation, that is, the position when the robot end is just in contact with the object but no squeezing occurs; : Contact normal deformation (unit: meter, m), representing the normal displacement generated by the robot end effector pressing against the object; : The tangential frictional force component vector (unit: Newton, N), calculated according to Coulomb's law of friction, has a magnitude not exceeding μ|kn(pc) p0)| (μ is the friction coefficient), the direction is opposite to the direction of the relative velocity at the contact point.

[0064] Normalization yielded the expected direction:

[0065] S242, Explicit Physical Feasibility Verification and Guidance.

[0066] calculate Geometry of sampling direction Match degree:

[0067] in, As expected, Let be the set of sampling directions, if , This is the tolerance threshold. If the current candidate action is determined to be valid within the geometric space, it is retained for robot execution. If the candidate action is not explicitly excluded, its logit is multiplied by 0 or a minimum value. Forced probability to be 0; or probability modulation: generation probability .

[0068] Alternatively, if explicit booting is not required, implicit booting can be used: Introducing a token modulation module, the structure is a lightweight MLP:

[0069] in, This is the generated physical modulation bias vector. A single-layer fully connected neural network specifically for token modulation, subscript This indicates its modulation function; For the contact cone geometry token corresponding to a single contact point; Here is the weight matrix of the token modulation module, with dimension 1. ; Here is the bias vector of the token modulation module, with dimension . D represents the embedding vector dimension, which is fixed at 512 in this invention to match the feature dimension of the VLA model. This modulation bias vector will be added to the attention score of each layer of the Transformer's attention mechanism, causing the model to naturally favor actions that satisfy the contact cone geometric constraints during autoregressive generation, thus achieving end-to-end implicit physical guidance.

[0070] In each attention layer (Multi-Head Attention) of the Transformer: Query The token is generated from the action; the key / value pair comes from the entire input sequence. Modulation of attention scores:

[0071] The modulated attention score matrix represents the degree of matching between the query vector and all key vectors in the Transformer attention layer. The query matrix is ​​obtained by linear transformation of the action tokens to be generated, with dimensions Nq×dk, where Nq is the number of query tokens. The key matrix is ​​obtained by linear transformation of all input sequences (visual tokens, language tokens, and contact cone geometric tokens), with dimensions Nk×dk, where Nk is the total number of input tokens. : Transpose of the key matrix; : Scaling factor, used to prevent the gradient of the softmax function from vanishing due to an excessively large dot product result; The physical modulation bias vector is generated by the token modulation module based on the contact cone geometry token, and has a dimension of 1× ; 1: A matrix consisting entirely of 1s; The physical modulation bias vector broadcast is extended into a bias matrix with the same dimension as the original attention score matrix, thereby achieving a unified physical bias for all attention scores.

[0072] ( Added as a bias broadcast to achieve physical feasibility bias).

[0073] Alternatively, use FiLM-style: scale and shift intermediate features (scale and shift element-wise).

[0074] in, : Modulated intermediate feature vector; : The original intermediate feature vector, which comes from the output of the previous layer of the Transformer; : The scaling factor vector is generated by passing the physical modulation bias vector m through an independent fully connected linear layer, and its dimension is the same as the original feature vector x; : The translation coefficient vector is generated by passing the physical modulation bias vector m through another independent fully connected linear layer, and its dimension is the same as the original feature vector x; The Hadamard product is the element-wise multiplication of corresponding positions of two vectors. ( From two linear layers generate).

[0075] Ultimately, this makes the policy distribution (probability after softmax) naturally biased towards actions that satisfy the contact cone, thus achieving preventative constraints.

[0076] S25. Update the contact information in real time, remodel the contact cone geometric constraints based on the updated contact information, and generate a new round of contact cone geometric tokens.

[0077] After the action is executed, the contact information (sensor feedback) is updated in real time, the above steps are re-executed, a new geometric token is generated, and closed-loop control is achieved.

[0078] In summary, the contact cone-based model robot guidance and control method provided by this invention proposes a contact cone geometry discrete sampling and encoding mechanism. This mechanism discretizes and structures the feasible domain of the contact cone direction into independent geometric tokens, transforming the mechanical geometric space into a structured input that can be processed by the higher-level VLA (Virtual Aspect-Based Logic) layer. A direct correlation is established between the contact cone geometric constraints and the action output in the VLA action generation stage, achieving a hybrid guidance of explicit verification and implicit modulation, which differs from existing force feedback fusion or low-level optimization methods. This invention achieves action generation and stable control for physically feasible robot guidance by structurally encoding the contact cone geometric constraints in contact mechanics through discrete direction sampling and introducing them into VLA.

[0079] Example 3 In another aspect, this invention provides a model robot guidance and control system based on a contact cone; please refer to [link / reference needed]. Figure 3 The diagram shows a schematic of the contact cone-based model robot guidance and control system in Embodiment 2 of the present invention. The contact cone-based model robot guidance and control system includes: The constraint modeling module 11 is used to acquire contact information, perform contact cone geometric constraint modeling based on the contact information, and divide the gripping safety zone. The token acquisition module 12 is used to perform geometric discrete sampling on the contact cone, obtain a set of sampling directions, map the set of sampling directions to an embedding vector, and concatenate the embedding vectors to obtain a contact cone geometric token. Action generation module 13 is used to fuse multimodal information and the contact cone geometric token, input the fusion result into a preset architecture, and generate an action sequence; The dynamic update module 14 is used to update the contact information in real time, remodel the contact cone geometric constraints based on the updated contact information, and generate a new round of contact cone geometric tokens.

[0080] Example 4 In another aspect, the present invention also proposes a computer-readable storage medium having stored thereon one or more computer programs that, when executed by a processor, implement the above-described contact cone-based model robot guidance and control method.

[0081] Those skilled in the art will understand that the logic or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable storage medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable storage medium" can mean any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0082] More specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable storage media can even be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0083] Example 5 Figure 4 This is a structural block diagram of an electronic device provided in Embodiment 4. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the contact cone-based model robot guidance and control method described in the above embodiments. Figure 4 The electronic device 30 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0084] like Figure 4 As shown, the electronic device 30 can be manifested as a general-purpose computing device, such as a server device. The components of the electronic device 30 may include, but are not limited to: at least one processor 31, at least one memory 32, and a bus 33 connecting different system components (including memory 32 and processor 31).

[0085] Bus 33 includes a data bus, an address bus, and a control bus.

[0086] The memory 32 may include volatile memory, such as RAM 321 (random access memory), and / or cache memory 322, and may further include ROM 323 (read-only memory).

[0087] The memory 32 may also include a program tool 325 having a set (at least one) of program modules 324, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0088] The processor 31 executes various functional applications and data processing by running computer programs stored in the memory 32, such as the contact cone-based model robot guidance and control method of the present invention as described above.

[0089] Electronic device 30 can also communicate with one or more external devices 34 (e.g., keyboard, pointing device, etc.). This communication can be performed via I / O interface 35 (input / output interface). Furthermore, electronic device 30 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public network, such as the Internet) via network adapter 36. Figure 3 As shown, network adapter 36 communicates with other modules of the model-generated electronic device 30 via bus 33. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the model-generated electronic device 30, including but not limited to: microcode, device drivers, redundant processors, disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems.

[0090] It should be noted that although several units / modules or sub-units / modules of the electronic device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0091] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0092] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this patent should be determined by the appended claims.

Claims

1. A contact cone based model robot guidance control method, characterized by, The methods include: Obtain contact information, perform contact cone geometric constraint modeling based on the contact information, and divide the gripping safety zone; Geometric discretization sampling is performed on the contact cone to obtain a set of sampling directions. The set of sampling directions is mapped to an embedding vector. The embedding vectors are concatenated to obtain a contact cone geometric token. By fusing multimodal information and the contact cone geometric token, the fusion result is input into a preset architecture to generate an action sequence; The contact information is updated in real time. Based on the updated contact information, the contact cone geometric constraints are remodeled, and a new round of contact cone geometric tokens is generated.

2. The model robot guidance and control method based on contact cone according to claim 1, characterized in that, In the steps of acquiring contact information, performing contact cone geometric constraint modeling based on the contact information, and dividing the gripping safety zone: Real-time acquisition of robot end-effector contact information, the contact information including at least the contact position. Contact normal vector and coefficient of friction ; Based on the contact information, a contact cone geometric model is constructed: in, The contact normal vector, This is the contact force vector.

3. The model robot guidance and control method based on contact cone according to claim 1, characterized in that, In the step of performing geometric discrete sampling on the contact cone to obtain the set of sampling directions: The sampling number of the contact cone geometric model is determined, the contact cone geometric model is divided based on the sampling number, the polar angle is determined simultaneously, and the local sampling direction is generated; The local sampling direction is transformed to world coordinates using a preset rotation matrix to obtain a set of sampling directions composed of several transformed local sampling directions.

4. The model robot guidance and control method based on contact cone according to claim 3, characterized in that, The steps after obtaining the sampling direction set are as follows: Any sampling direction in the normalized set of sampling directions is mapped to an embedding vector of fixed dimension, wherein the embedding vector contains position encoding; A preset number of embedding vectors are concatenated, and a single independent geometric token is obtained through linear projection. The preset number is equal to the number of samples. The single independent geometric token is an independent geometric token under a single contact point. Several independent geometric tokens under a single contact point form a geometric token sequence with multiple contact points.

5. The model robot guidance and control method based on contact cone according to claim 1, characterized in that, The steps for fusing multimodal information and the contact cone geometric token, inputting the fusion result into a preset architecture, and generating an action sequence are as follows: The system receives visual information encoded by a visual encoder, instruction information encoded by a language encoder, and a geometric token sequence, respectively, and fuses them to generate a unified input sequence. The unified input sequence is then input into a preset VLA model structure to obtain an autoregressive action sequence.

6. The model robot guidance and control method based on contact cone according to claim 5, characterized in that, Based on the candidate actions in the autoregressive generated action sequence, the robot end-effector posture at the next moment is predicted by the forward dynamics module. The contact point velocity and expected contact force are calculated using the predicted robot end-effector posture, and the expected direction is obtained by normalization. Calculate the matching degree between the expected direction and the set of sampling directions: in, As expected, Let be the set of sampling directions, if , If the tolerance threshold is met, then within the geometric space, the current candidate action is retained for robot execution. If so, then explicit exclusion will be performed.

7. A model robot guidance and control system based on a contact cone, characterized in that, The contact cone-based model robot guidance and control system is used to implement the contact cone-based model robot guidance and control method according to any one of claims 1-6, the system comprising: The constraint modeling module is used to acquire contact information, perform contact cone geometric constraint modeling based on the contact information, and divide the gripping safety zone. The token acquisition module is used to perform geometric discrete sampling on the contact cone, obtain a set of sampling directions, map the set of sampling directions to an embedding vector, and concatenate the embedding vectors to obtain a contact cone geometric token. An action generation module is used to fuse multimodal information and the contact cone geometric token, input the fusion result into a preset architecture, and generate an action sequence. The dynamic update module is used to update the contact information in real time, remodel the contact cone geometric constraints based on the updated contact information, and generate a new round of contact cone geometric tokens.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the contact cone-based model robot guidance and control method as described in any one of claims 1-6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the contact cone-based model robot guidance and control method as described in any one of claims 1-6.

Images