Reachable probability calculation model training method, control method, device and storage medium

Abstract

Embodiments of the present application disclose a reachable probability calculation model training method, a control method, a device and a storage medium. The training method comprises the following steps: determining to-be-screened end pose data according to design parameters of a mechanical arm; selecting reachable end pose data from the to-be-screened end pose data and generating a training set; training a multivariate Gaussian mixture model for fitting the reachable pose data space distribution based on the training set, and generating a reachable probability calculation model based on the multivariate Gaussian mixture model, the reachable probability calculation model being used to output a reachable probability corresponding to any end pose. Therefore, the present application can construct and train the calculation model by adopting the architecture of the multivariate Gaussian mixture model. When implementing control, the reachable probability is calculated by the model to control, so as to avoid failure caused by "able to grasp but unreachable", and improve the operation success rate of the mechanical arm in a complex environment.

Description

Technical Field

[0001] This application belongs to the field of mechanical control technology, and in particular relates to a method for training an reachability probability calculation model, a robotic arm control method, a computer device, and a computer-readable storage medium. Background Technology

[0002] In daily life, robotic arms have been widely used in industrial automation, medical surgery assistance, and home services. Their operational precision and response speed directly affect the quality of task completion. Currently, the control of robotic arms for grasping and placing objects mainly falls into three categories: analytical geometry-based "force-closure" or "shape-closure" methods (such as GraspIt!), end-to-end methods based on 2D / 3D convolutional networks (such as GG-CNN and GR-ConvNet), and 6-DoF pose estimation networks based on point clouds or RGB-D (such as GSNet, Any Grasp, and Contact-GraspNet).

[0003] However, current technologies for robotic arm grasping and placement tasks typically simplify it to "placing an object on a plane." Therefore, the control of the robotic arm focuses solely on whether it can grasp and place the object successfully. Most existing robotic arm grasping methods only consider how to generate the grasping pose, rarely addressing the accessibility of reaching that pose. Even when they do consider it, it's often a simplistic, binary approach, such as using the residual from solving inverse kinematics followed by forward kinematics, or the number of iterations in the inverse kinematics iteration as the optimization target loss value. This results in slow solutions and poor real-time performance. How to more efficiently control robotic arm grasping is a technical problem that urgently needs to be solved by those skilled in the art.

[0004] The preceding description is intended to provide general background information and does not necessarily constitute prior art. Summary of the Invention

[0005] The purpose of this application is to provide a method for training an reachability probability calculation model, a robotic arm control method, a computer device, and a computer-readable storage medium capable of calculation.

[0006] To achieve the above objectives:

[0007] In a first aspect, embodiments of this application provide a method for training an reachability probability calculation model, comprising the following steps: randomly generating a preset number of joint state combinations according to the design parameters of the robotic arm, and determining the end-effector pose data to be screened for each joint state combination through forward kinematics solution; determining the unreachable end-effector pose data in the end-effector pose data to be screened, removing the unreachable end-effector pose data from the end-effector pose data to be screened to obtain reachable end-effector pose data, and generating a training set based on the reachable end-effector pose data; training a multivariate Gaussian mixture model for fitting the spatial distribution of reachable pose data based on the training set, and generating an reachability probability calculation model based on the multivariate Gaussian mixture model, wherein the reachability probability calculation model is used to output the reachability probability corresponding to any end-effector pose.

[0008] In an optional embodiment of this application, determining unreachable end-effector pose data from the end-effector pose data to be screened includes: acquiring the self-collision condition constraints of the robotic arm for joint state combinations, and determining the end-effector pose data to be screened corresponding to joint state combinations that do not satisfy the self-collision constraints from a preset number of joint state combinations as unreachable end-effector pose data; and / or, acquiring the task workspace corresponding to the robotic arm under the current task, wherein the task workspace is the neighborhood space around the target object corresponding to the current task; determining the end-effector pose data to be screened that are not in the neighborhood space as unreachable end-effector pose data; and / or, acquiring the obstacle point cloud in the external environment of the robotic arm; determining the end-effector pose data to be screened corresponding to the joint state combinations in a preset number of joint state combinations where the robotic arm point cloud obtained based on the design parameters collides with the obstacle point cloud as unreachable end-effector pose data.

[0009] In an optional embodiment of this application, generating a training set based on reachable end pose data includes: performing standard normalization processing on the reachable end pose data to obtain first data; performing an anomaly removal operation on the first data using a preset screening method to remove abnormal data in the first data, and marking the remaining data as second data; generating samples based on the second data, and summarizing all samples to obtain a training set.

[0010] In an optional embodiment of this application, training a multivariate Gaussian mixture model for fitting the spatial distribution of reachable pose data based on a training set includes: training the model according to a preset training method based on the training set to obtain training parameters, which include mixture weights, mean vector, and covariance matrix; constructing a multivariate Gaussian mixture model based on the training parameters, wherein the multivariate Gaussian mixture model takes pose data as input and processes it to obtain the log-likelihood value corresponding to the pose data, and the log-likelihood value is used to describe the spatial probability distribution density of the pose data.

[0011] In an optional embodiment of this application, generating an reachability probability calculation model based on a multivariate Gaussian mixture model includes: inputting all samples in the training set into the multivariate Gaussian mixture model to obtain the log-likelihood values ​​of all samples; determining the 99th percentile of the log-likelihood values ​​of all samples based on the log-likelihood values ​​of all samples, and marking the 99th percentile as a reference standard value; obtaining a preset temperature coefficient; generating an reachability probability calculation model based on the temperature coefficient, the reference standard value, and the multivariate Gaussian mixture model, wherein the reachability probability calculation model takes the end pose as input, processes the end pose according to the multivariate Gaussian mixture model to obtain the log-likelihood value, and calculates the corresponding reachability probability based on the log-likelihood value, the temperature coefficient, and the reference standard value.

[0012] Secondly, embodiments of this application provide a robotic arm control method, including the following steps: when a control command is received, at least one candidate target pose is calculated and determined according to the control command; the candidate target pose is input into an reachability probability calculation model for processing to obtain the reachability probability corresponding to the candidate target pose, wherein the reachability probability calculation model is trained by the method provided above; the optimal target pose is selected from the candidate target poses according to the reachability probability; and the robotic arm is controlled to move to the target position corresponding to the control command with the optimal target pose.

[0013] In an optional embodiment of this application, the candidate target pose is input into the reachability probability calculation model for processing to obtain the reachability probability corresponding to the candidate target pose. This includes: processing the candidate target pose into a log-likelihood value according to the multivariate Gaussian model mixture model in the reachability probability calculation model, wherein the multivariate Gaussian model mixture model is trained by the method provided above; and calculating the corresponding reachability probability based on the log-likelihood value and a preset temperature coefficient and a reference standard value, wherein the temperature coefficient is a preset value and the reference standard value is obtained by training the reachability probability calculation model.

[0014] In an optional embodiment of this application, the method further includes: inputting the candidate target pose into the reachability probability calculation model for processing to obtain the reachability probability corresponding to the candidate target pose; marking the probability output by the reachability probability calculation model as a first reachability probability; obtaining at least one pre-constructed and trained auxiliary calculation model; inputting the candidate target pose into the at least one auxiliary calculation model respectively to obtain at least one second reachability probability output by each of the at least one auxiliary calculation model; and determining the reachability probability corresponding to the candidate target pose based on the first reachability probability and the at least one second reachability probability.

[0015] In an optional embodiment of this application, at least one reachability probability calculation model includes a first auxiliary calculation model. The first auxiliary calculation model is used to: determine the workspace and reachable end-effector pose data of the robotic arm according to the design parameters of the robotic arm; divide the workspace into multiple discrete units, each discrete unit representing a three-dimensional space of the same preset size in the workspace; determine the reachability probability corresponding to each discrete unit based on the number of reachable end-effector pose data contained in each discrete unit; obtain the candidate target pose in the workspace, determine the target discrete unit in which the candidate target pose is located, and determine the reachability probability corresponding to the target discrete unit as the second reachability probability corresponding to the candidate target pose.

[0016] In an optional embodiment of this application, at least one reachability probability calculation model includes a second auxiliary calculation model. The second auxiliary calculation model is used to: input the candidate target pose into a trained autoencoder model, obtain the reconstruction error value of the candidate target pose under the autoencoder model, wherein the autoencoder model is trained based on the training set and is used to generate and parse the latent representation corresponding to the reachable end pose data; calculate the second reachability probability output by the second auxiliary calculation model based on the reconstruction error value, the reference standard value and the preset temperature parameter, wherein the reference standard value is the 99th percentile of the reconstruction error values ​​corresponding to all samples in the training set.

[0017] Thirdly, embodiments of this application provide a computer device, including: a processor and a memory storing a computer program, wherein when the processor runs the computer program, the steps of the above-described method are implemented.

[0018] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0019] The embodiments of this application have the following beneficial effects:

[0020] The reachability probability calculation model training method provided in this application includes the following steps: Based on the design parameters of the robotic arm, a preset number of joint state combinations are randomly generated, and the end-effector pose data corresponding to each joint state combination is determined through forward kinematics solving; unreachable end-effector pose data is determined from the end-effector pose data to be screened, and reachable end-effector pose data is obtained by removing the unreachable end-effector pose data from the end-effector pose data to be screened, and a training set is generated based on the reachable end-effector pose data; a multivariate Gaussian mixture model for fitting the spatial distribution of reachable pose data is trained based on the training set, and a reachability probability calculation model is generated based on the multivariate Gaussian mixture model, which is used to output the reachability probability corresponding to any end-effector pose. The provided robotic arm control method includes the following steps: When a control command is received, at least one candidate target pose is calculated and determined according to the control command; the candidate target pose is input into the reachability probability calculation model for processing to obtain the reachability probability corresponding to the candidate target pose, and the reachability probability calculation model is trained by the method provided above; the optimal target pose is selected from the candidate target poses based on the reachability probability; the robotic arm is controlled to move to the target position corresponding to the control command with the optimal target pose. Therefore, this application can construct and train a computational model based on the parameters of the robotic arm using a multivariate Gaussian mixture model. The multivariate Gaussian mixture model is trained on the training set and used to fit the spatial distribution of the candidate target pose. The trained model is labeled as a reachability probability calculation model, which outputs the reachability probability corresponding to any end-effector pose. Controlling the robotic arm through reachability probability, unlike the binary judgment of whether the robotic arm is reachable in existing technologies, enables more effective and precise control.

[0021] The above description is merely an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it according to the contents of the specification, and to make the above and other objects, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and do not limit this application. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating a training method for an reachability probability calculation model, provided as an embodiment.

[0024] Figure 2 This is a flowchart illustrating a robotic arm control method provided in one embodiment.

[0025] Figure 3 This is a schematic block diagram of the structure of a computer device provided in one embodiment. Detailed Implementation

[0026] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. In the following description relating to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements.

[0027] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, components, features, and elements with the same names in different embodiments of this application may have the same meaning or different meanings, the specific meaning of which must be determined by its interpretation in that specific embodiment or further in conjunction with the context of that specific embodiment.

[0028] It should be understood that although the terms first, second, third, etc., may be used herein to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this document, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if," as used herein, can be interpreted as "when," "when," or "in response to determination." Furthermore, as used herein, the singular forms "a," "an," and "the" are intended to also include the plural forms unless the context indicates otherwise. It should be further understood that the terms "comprising," "including," indicate the presence of the stated feature, step, operation, element, component, item, kind, and / or group, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, kinds, and / or groups. The terms "or" and "and / or" as used herein are to be interpreted as inclusive, or mean any one or any combination thereof. Therefore, "A, B, or C" or "A, B, and / or C" means "any one of the following: A; B; C; A and B; A and C; B and C; A, B, and C". Exceptions to this definition will only occur if the combination of elements, functions, steps, or operations is inherently mutually exclusive in some way.

[0029] It should be understood that although the steps in the flowcharts of this application's embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0030] It should be noted that step designations such as S110 and S120 are used in this document for the purpose of more clearly and concisely describing the corresponding content, and do not constitute a substantial limitation on the order. In specific implementation, those skilled in the art may execute S120 first and then S110, etc., but these should all be within the protection scope of this application.

[0031] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0032] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.

[0033] To facilitate understanding of this application, the following explanations are provided for the terms and technical objects that may be used in this application:

[0034] Reachable workspace: The set of origin positions of the end effector coordinate system when all joints of the robotic arm are performing all possible movements.

[0035] Probabilistic reachability workspace: Based on the reachability workspace, probability values ​​are used to characterize the ease with which the end-effector can reach a certain (or a certain type of) pose. The larger the value, the higher the reachability.

[0036] The current mainstream approaches to grasping pose generation can be roughly divided into three categories: (1) "force closure" or "shape closure" methods based on analytical geometry (such as GraspIt!), which calculate the closure through the contact point normal and friction cone, require a known precise geometric model, have a large computational load, and are difficult to implement in real time. (2) End-to-end methods based on 2D / 3D convolutional networks (such as GG-CNN, GR-ConvNet), which directly output pixel-by-pixel grasping confidence maps, are fast but only provide 4-DoF (x,y,z,yaw) grasping, lack roll / pitch constraints, and are prone to conflict with the limits of the robotic arm joints. (3) 6-DoF pose estimation networks based on point clouds or RGB-D (such as GSNet, AnyGrasp, Contact-GraspNet), which first detect the graspable region and then regress the 6D pose, have high accuracy, but still assume that "as long as there is a grasping pose, the movement can reach it", without systematically evaluating accessibility.

[0037] Reachability verification and replanning in existing solutions typically employ a two-step approach: Step 1: Call numerical inverse kinematics (IK) tools such as IKFast, BioIK, and Trac-IK on candidate poses; if an empty return value is received, the pose is considered unreachable. Step 2: If IK is successful, use motion planners such as RRT-Connect, CHOMP, and STOMP to check for collision-free trajectories. However, existing technologies have the following drawbacks: 1. Each candidate pose requires an independent IK+ planning call, resulting in high latency when there are many candidates (>1k); 2. Only a binary result of "feasible / infeasible" is provided, failing to quantify "ease of reach"; 3. When the environment changes slightly (e.g., object displacement of 2cm), all calculations must be recalculated, lacking online adaptive capabilities.

[0038] The task of placing a robotic arm is often simplified to "placing an object in a certain place on a plane". Existing technologies adopt the following approaches: 1. Rule-based method: offsetting by a safe height along the normal of the support surface, randomly sampling the translation position, without pose optimization; 2. Learning method: such as PoseCNN, 6-PACK, etc. first estimate the object pose, and then take the inverse pose as the placement pose, without considering the reachable space of the robotic arm end effector in the placement area; 3. Constraint method: such as ReKep, KeyPose, etc. describe the placement relationship through key point constraints, but constraint solving still requires multiple IK iterations, resulting in poor real-time performance.

[0039] For probabilistic / learning-based reachability space modeling (a few exploratory works), existing techniques employ the following approaches, each with its own limitations. "Reachability Map" (Vahrenkamp et al., IJRR 2013) constructs a 3D reachability probability map using grid counting, but only considers position, neglecting pose. "Reachability GMM" (Kim et al., ICRA2021) fits a 6D end-effector distribution using a Gaussian mixture model, but suffers from small training data (<1M), low resolution, and difficulty in finely characterizing narrow spaces. "Neural Reachability Fields" (Wang et al., RSS 2023) represents feasible joint manifolds using implicit neural fields, requires GPUs for inference, and is not deeply coupled with grasping / placement tasks.

[0040] In summary, existing robotic arm control methods mostly focus on generating the grasping pose, with less consideration given to the accessibility of reaching that pose. Even when accessibility is considered, it's often simplistic, such as using the residual from solving the inverse kinematics and then the forward kinematics, or the number of iterations in the inverse kinematics iteration as the optimization target loss value. This results in slow solutions and poor real-time performance. They either ignore accessibility entirely or only perform posterior verification, lacking a closed-loop mechanism that considers accessibility probability simultaneously with grasping / placement pose generation. This leads to low overall success rate, poor real-time performance, and weak adaptability to dynamic environments. Therefore, this paper proposes a method for training an accessibility probability calculation model and a robotic arm control method to overcome the shortcomings of existing technologies. For a clear description of the method provided in this embodiment, please refer to [link to relevant documentation]. Figures 1-2 This includes steps S110~S130 and steps S210~S240.

[0041] Step S110: Based on the design parameters of the robotic arm, a preset number of joint state combinations are randomly generated, and the end pose data to be screened corresponding to each joint state combination is determined by forward kinematics solution.

[0042] In one embodiment, the design parameters of the robotic arm may specifically include the link length, joint type, range of motion, and drive method. Based on the robotic arm's degrees of freedom and joint constraints, Monte Carlo simulation is used to generate a large number of joint state combinations, each encompassing the pose states of each joint of the robotic arm. For example, for a 7-DOF robotic arm, 10 random samples are generated within the joint constraints. 7Groups of joint states are combined. Each group of joint state combinations is solved using forward kinematics to obtain the corresponding end-effector pose data to be filtered. This end-effector pose data is obtained by directly solving the joint state combinations and is used to indicate the end-effector pose of the robotic arm under the corresponding joint parameter settings. Furthermore, for the generation of end-effector pose data, feasible joint states can be generated using sampling planners such as RRT* and PRM to replace random sampling, which can improve coverage efficiency under complex joint constraints. Simultaneously, a GPU-parallelized batch IK solver is used to back-map the end-effector pose to joint space, and then infeasible samples are filtered out.

[0043] Step S120: Determine the unreachable end pose data in the end pose data to be screened, remove the unreachable end pose data from the end pose data to be screened to obtain reachable end pose data, and generate a training set based on the reachable end pose data.

[0044] In one embodiment, determining unreachable end-effector pose data from the end-effector pose data to be screened includes: acquiring self-collision constraints for joint state combinations of the robotic arm; identifying the end-effector pose data corresponding to joint state combinations that do not satisfy the self-collision constraints from a preset number of joint state combinations as unreachable end-effector pose data; and / or acquiring the task workspace corresponding to the robotic arm in the current task, wherein the task workspace is the neighborhood space around the target object corresponding to the current task; identifying the end-effector pose data not in the neighborhood space as unreachable end-effector pose data; and / or acquiring obstacle point clouds in the external environment of the robotic arm; identifying the end-effector pose data corresponding to joint state combinations in a preset number of joint state combinations where the robotic arm point cloud obtained based on design parameters collides with the obstacle point cloud as unreachable end-effector pose data.

[0045] In one embodiment, it is understood that not all of the generated large amount of end-effector pose data is usable for the robotic arm; many of these end-effector pose data are unreachable or would cause collisions during movement. Therefore, it is necessary to filter the end-effector pose data to remove the unreachable data. This application provides three optional filtering methods.

[0046] The process involves obtaining self-collision constraints for joint state combinations of the robotic arm, and identifying the end-effector pose data corresponding to joint state combinations that do not meet the self-collision constraints from a preset number of joint state combinations as unreachable end-effector pose data. Self-collision constraints are obtained, such as joint torque limits, obstacle avoidance requirements, or path continuity, or the generated joint states are determined based on the robot / robotic arm's URDF file to determine whether they will collide with themselves. Pose combinations that will collide with themselves are marked as unreachable. Finally, the end-effector pose data corresponding to the joint state combinations that do not meet the self-collision constraints are identified as unreachable end-effector pose data.

[0047] The task workspace of the robotic arm under the current task is obtained. The task workspace is the neighborhood space surrounding the target object corresponding to the current task. End-effector pose data not located in the neighborhood space are identified as unreachable. The task workspace is the neighborhood space surrounding the target object of the current task being performed by the robotic arm, which is also the destination the robotic arm needs to reach. End-effector pose data not located in the neighborhood space is clearly considered unreachable and cannot fulfill the corresponding task requirements.

[0048] Obtain obstacle point clouds from the external environment of the robotic arm; identify the unreachable end-effector pose data corresponding to joint state combinations where the robotic arm point cloud (reconstructed based on design parameters) collides with the obstacle point clouds from a preset number of joint state combinations. The obstacle point cloud describes the distribution of static obstacles, dynamic interference areas, and spatial constraints in the robotic arm's working environment. Through the obstacle point cloud, pose data where the robotic arm may collide with the working environment can be determined. These unreachable end-effector pose data are then identified as unreachable end-effector pose data.

[0049] It's worth noting that the obstacle point cloud removal method prioritizes static scenes, i.e., situations where the robotic arm's base doesn't move. In this case, the relative spatial relationship between the robotic arm and the environment is relatively clear, facilitating accurate calculation of collision relationships through pre-modeling. If the robotic arm moves during operation, environmental and target object parameters need to be updated in real-time in a dynamic environment. For this, a spatiotemporal reachability probability model can be introduced, incorporating predictions of future environmental states into probability calculations to dynamically assess reachability.

[0050] The unreachable end pose data determined by the above screening method are removed from the end pose data to be screened to obtain reachable end pose data.

[0051] The training set is generated based on the reachable end pose data, specifically including: performing standard normalization on the reachable end pose data to obtain the first data; performing anomaly removal operation on the first data using a preset filtering method to remove abnormal data in the first data, and marking the remaining data as the second data; generating samples based on the second data, and summarizing all samples to obtain the training set.

[0052] In one implementation, the reachable end-effector pose data is standardized and normalized. For example, the reachable end-effector pose data is processed into a 6-dimensional end-effector pose, where translation includes three dimensions (x, y, z) and rotation includes three dimensions (Eulerian angles: rx, ry, rz). For ease of processing, the end-effector pose data can be converted into feature data, then normalized to the same scale or key statistical features can be extracted to obtain the first data. An anomaly removal operation is performed on the first data using a preset screening method to determine whether there are discrete outliers, etc. The outliers are filtered out from the first data, and the remaining data is marked as the second data. Finally, samples are constructed based on the second data, with each second data item corresponding to one sample. All samples are then aggregated to obtain the training set.

[0053] Step S130: Train a multivariate Gaussian mixture model based on the training set to fit the spatial distribution of reachable pose data, and generate an reachability probability calculation model based on the multivariate Gaussian mixture model. The reachability probability calculation model is used to output the reachability probability corresponding to any end pose.

[0054] In one implementation, a multivariate Gaussian mixture model for fitting the spatial distribution of reachable pose data is trained based on a training set, including: training the training set according to a preset training method to obtain training parameters, which include mixture weights, mean vector, and covariance matrix; constructing a multivariate Gaussian mixture model based on the training parameters, wherein the multivariate Gaussian mixture model takes pose data as input and processes it to obtain the log-likelihood value corresponding to the pose data, and the log-likelihood value is used to describe the spatial probability distribution density of the pose data.

[0055] In one implementation, the generation of the training set has been described in detail above. The subsequent step is to train the multivariate Gaussian mixture model. This training process can be implemented using existing training algorithms, such as the EM (Expectation-Maximization) algorithm. The purpose of training is to determine the training parameters, including the mixture weights, mean vector, and covariance matrix. Based on the training parameters, a multivariate Gaussian mixture model can be constructed, and its mathematical expression is as follows.

[0056] (1)

[0057] In the above formula, It is the weight of the k-th Gaussian component; It is the mean vector of this component, corresponding to a "high-density" region in the reachable space; The covariance matrix describes the shape and orientation of the local region. The multivariate Gaussian mixture model takes pose data as input and processes it to obtain the log-likelihood value corresponding to the pose data. The log-likelihood value is used to describe the spatial probability distribution density of the pose data.

[0058] In one implementation, generating an reachability probability calculation model based on a multivariate Gaussian mixture model includes: inputting all samples in the training set into the multivariate Gaussian mixture model to obtain the log-likelihood values ​​of all samples; determining the 99th percentile of the log-likelihood values ​​of all samples based on the log-likelihood values ​​of all samples, and marking the 99th percentile as a reference standard value; obtaining a preset temperature coefficient; generating an reachability probability calculation model based on the temperature coefficient, the reference standard value, and the multivariate Gaussian mixture model, wherein the reachability probability calculation model takes the end pose as input, processes the end pose according to the multivariate Gaussian mixture model to obtain the log-likelihood value, and calculates the corresponding reachability probability based on the log-likelihood value, the temperature coefficient, and the reference standard value.

[0059] In one implementation, a multivariate Gaussian mixture model is used to process the input pose data into log-likelihood values. Since the reachability probability calculation model aims to calculate the reachability probability, it can be constructed to achieve this calculation. First, all samples in the training set are input into the multivariate Gaussian mixture model, and the log-likelihood values ​​of all samples are calculated. The 99th quantile of the log-likelihood values ​​of all samples is then determined, and this 99th quantile is marked as the reference standard value. The reference standard value can be calculated using the following formula.

[0060] (2)

[0061] Obtain the preset temperature coefficient, and generate an reachability probability calculation model based on the temperature coefficient, reference standard value, and multivariate Gaussian mixture model. The mathematical model of the reachability probability calculation model can be referred to in the following formula.

[0062] (3)

[0063] In the above formula, This is a temperature parameter that controls the smoothness of the probability output. A smaller value results in a more stringent model judgment; the default value must be greater than 0. (Function) The calculation method is shown in the following formula.

[0064] (3)

[0065] The reachability probability calculation model takes the end-effector pose as input and processes it using a multivariate Gaussian mixture model to obtain a log-likelihood value. The reachability probability is then calculated based on the log-likelihood value, the temperature coefficient, and a reference standard value. The reachability probability calculated by the reachability probability calculation model is explained as follows: When... hour, ;when hour, ;when hour, .

[0066] This completes the construction and training of the reachability probability calculation model.

[0067] Correspondingly, this application also proposes a robotic arm control method, for details please refer to Figure 2 This includes steps S210 to S240.

[0068] Step S210: When a control command is received, at least one candidate target pose is calculated and determined according to the control command.

[0069] In one embodiment, the control command is the instruction to control the robotic arm to move to a specific space in a specific pose to grasp or place a target object. For the control command, at least one candidate target pose can be generated based on 6D grasping pose estimation methods such as GSNet, Anygrasp, and RNGgrasp. A nearest neighbor region is selected at the target's center location, and the target pose is expanded slightly. Combining the positions of these nearest neighbor regions and the expanded pose yields a large number of placement positions. This candidate target pose contains pose data, meaning it includes not only where the robotic arm needs to go but also the posture in which it should go.

[0070] Step S220: Input the candidate target pose into the reachability probability calculation model for processing to obtain the reachability probability corresponding to the candidate target pose. The reachability probability calculation model is trained by the method provided above.

[0071] In one embodiment, the candidate target pose is input into the reachability probability calculation model for processing to obtain the reachability probability corresponding to the candidate target pose. This includes: processing the candidate target pose into a log-likelihood value according to the multivariate Gaussian mixture model in the reachability probability calculation model, wherein the multivariate Gaussian mixture model is trained by the method provided above; and calculating the corresponding reachability probability based on the log-likelihood value and a preset temperature coefficient and a reference standard value, wherein the temperature coefficient is a preset value and the reference standard value is obtained by training the reachability probability calculation model.

[0072] In one embodiment, the reachability probability calculation model is trained using the method described above. The reachability probability calculation model takes the end-effector pose as input and processes the end-effector pose according to a multivariate Gaussian mixture model to obtain a log-likelihood value. The reachability probability is then calculated based on the log-likelihood value, the temperature coefficient, and the reference standard value. The calculation process is as follows: First, the candidate target pose is processed into a log-likelihood value according to the multivariate Gaussian mixture model in the reachability probability calculation model. For the specific calculation method, please refer to formula (1). Subsequently, the reachability probability is calculated based on the log-likelihood value, the preset temperature coefficient, and the reference standard value. For the specific calculation method, please refer to formula (3). Based on this, the reachability probabilities of multiple candidate target poses are calculated to describe whether the robotic arm can reach the target position and whether it is difficult to reach the target position.

[0073] In one embodiment, the candidate target pose is input into an reachability probability calculation model for processing to obtain the reachability probability corresponding to the candidate target pose. The method further includes: marking the probability output by the reachability probability calculation model as a first reachability probability; obtaining at least one pre-constructed and trained auxiliary calculation model, and inputting the candidate target pose into the at least one auxiliary calculation model respectively to obtain at least one second reachability probability output by each of the at least one auxiliary calculation model; and determining the reachability probability corresponding to the candidate target pose based on the first reachability probability and the at least one second reachability probability.

[0074] In one implementation, the probability output by the reachability probability calculation model is labeled as the first reachability probability. Alternatively, at least one pre-built and trained auxiliary calculation model can be used to process the candidate target pose to obtain a corresponding second reachability probability. The second reachability probability is the reachability probability obtained under other calculation models. Finally, the specific reachability probability can be determined by weighted summation, multiplication, or a combination of both, balancing real-time performance and accuracy, and can be flexibly switched according to hardware conditions. The reachability probability is used to evaluate the quality score of the pose taken by the robotic arm when reaching the target position, that is, it represents the reachability and stability of the pose in the current environment. The higher the score, the closer the robotic arm is to the ideal motion state when reaching the target with that pose, the more balanced the joint forces, the stronger the obstacle avoidance ability, and the smoother the overall execution.

[0075] In one embodiment, at least one reachability probability calculation model includes a first auxiliary calculation model, which is used to: determine the workspace and reachable end-effector pose data of the robotic arm according to the design parameters of the robotic arm; divide the workspace into multiple discrete units, each discrete unit representing a three-dimensional space of the same preset size in the workspace; determine the reachability probability corresponding to each discrete unit based on the number of reachable end-effector pose data contained in each discrete unit; obtain candidate target poses in the workspace, determine the target discrete unit in which the candidate target pose is located, and determine the reachability probability corresponding to the target discrete unit as the second reachability probability corresponding to the candidate target pose.

[0076] In one embodiment, the workspace of the robotic arm is determined based on its design parameters. For example, the scene requirements in the design parameters are obtained. These scene requirements indicate the environmental layout, target object position, and working range for the robotic arm to perform specific tasks. Based on this, the workspace of the robotic arm, i.e., the boundary of its movement, is defined.

[0077] The reachability probability is essentially a quantitative assessment of the reachability of the robotic arm at each end-effector pose within the workspace, including whether it can reach the target position and how easily it can be reached. In other words, each pose in the workspace corresponds to a reachability probability value. Therefore, a large number of candidate grasping poses can be generated based on 6D grasping pose estimation methods such as GSNet, Anygrasp, and RNGgrasp. Each candidate grasping pose has a grasping quality evaluation score, which can be represented in the form of reachability probability.

[0078] In this application, the reachability probability of a robotic arm is statistically determined using discrete voxels. The 3D data network in the workspace is divided into a 3D mesh composed of equally spaced voxels, with each voxel corresponding to a discrete unit, which is also a 3D space of a preset size. Each discrete unit corresponds to a spatial location, and each sample corresponding to reachable pose data also corresponds to a spatial location. Based on the mapping relationship of these spatial locations, the number of reachable end-effector pose data contained in the 3D space corresponding to each discrete unit can be determined. The number of reachable end-effector pose data in each discrete unit is denoted as the cardinality; the maximum value of all cardinality values ​​is denoted as the baseline. The reachability probability corresponding to each discrete unit can be determined based on the ratio of the cardinality corresponding to each discrete unit to the baseline.

[0079] In one embodiment, at least one reachability probability calculation model includes a second auxiliary calculation model, which is used to: input the candidate target pose into a trained autoencoder model, obtain the reconstruction error value of the candidate target pose under the autoencoder model, wherein the autoencoder model is trained based on a training set and is used to generate and parse the latent representation corresponding to the reachable end pose data; and calculate the second reachability probability output by the second auxiliary calculation model based on the reconstruction error value, a reference standard value and a preset temperature parameter, wherein the reference standard value is the 99th percentile of the reconstruction error values ​​corresponding to all samples in the training set.

[0080] In one embodiment, an autoencoder model architecture can be adopted for the second auxiliary computational model. The process for calculating the second reachability probability in the second auxiliary computational model under the autoencoder model architecture is as follows: The candidate target pose input is processed to obtain the corresponding reconstruction error value, calculated using the following formula.

[0081] (4)

[0082] In the above formula, Input candidate target pose. The corresponding reconstructed values ​​obtained by processing candidate target poses for the autoencoder model; To reconstruct the error values, the autoencoder model is trained on a training set and used to generate and parse the latent representations corresponding to the candidate target poses. This training set is the one generated earlier using the reachable end-effector pose data.

[0083] The second reachability probability output by the second auxiliary calculation model is obtained by calculating based on the reconstruction error value, the reference standard value and the preset temperature parameters. The calculation method is as follows.

[0084] (5)

[0085] In the above formula, This serves as the reference standard value for the second auxiliary calculation model, specifically the 99th percentile of the reconstruction error value corresponding to all samples in the training set during the training process of the second auxiliary calculation model.

[0086] Step S230: Select the optimal target pose from the candidate target poses based on the reachability probability.

[0087] In one implementation, each candidate target pose has a corresponding reachability probability after retrieval. The candidate target poses are then sorted according to their reachability probabilities. The optimal target position is selected from these sorted poses; for example, the candidate target pose with the highest reachability probability is chosen as the optimal target position.

[0088] Step S240: Control the robotic arm to move to the target position corresponding to the control command in the optimal target pose.

[0089] In one implementation, the robotic arm can then be controlled to move to the optimal target position in the optimal target pose. As mentioned earlier, the optimal target position not only refers to moving to a specific spatial location, but also includes the pose with which it moves. The "reachability probability" is explicitly introduced into the grasping / placement scoring system to avoid failures caused by "being able to grasp but not being able to reach."

[0090] Therefore, the reachability probability calculation model training method provided in this application includes the following steps: Based on the design parameters of the robotic arm, a preset number of joint state combinations are randomly generated, and the end-effector pose data corresponding to each joint state combination is determined through forward kinematics solution; unreachable end-effector pose data are determined from the end-effector pose data to be screened, and reachable end-effector pose data is obtained after removing the unreachable end-effector pose data from the end-effector pose data to be screened, and a training set is generated based on the reachable end-effector pose data; a multivariate Gaussian mixture model for fitting the spatial distribution of reachable pose data is trained based on the training set, and a reachability probability calculation model is generated based on the multivariate Gaussian mixture model, which is used to output the reachability probability corresponding to any end-effector pose. The provided robotic arm control method includes the following steps: When a control command is received, at least one candidate target pose is calculated and determined according to the control command; the candidate target pose is input into the reachability probability calculation model for processing to obtain the reachability probability corresponding to the candidate target pose, and the reachability probability calculation model is trained by the method provided above; the optimal target pose is selected from the candidate target poses based on the reachability probability; the robotic arm is controlled to move to the target position corresponding to the control command with the optimal target pose. Therefore, this application can construct and train a computational model based on the multivariate Gaussian mixture model (MMM) architecture for the robotic arm parameters. The MMM is trained on the training set and used to fit the spatial distribution of the candidate target pose. The trained model is labeled as the reachability probability calculation model, which outputs the reachability probability corresponding to any end-effector pose. Controlling the robotic arm through reachability probability, unlike the binary judgment of whether the robotic arm is reachable in existing technologies, enables more effective and precise control. This application introduces a probabilistic evaluation mechanism, comprehensively considering factors such as the robotic arm's structural design, interference from the working environment, and the influence of the grasped object and the environment, to determine the credibility of the robotic arm reaching the target position in actual operation, i.e., whether it can be reached, how easily it can be reached, and what pose is more likely to reach it. Furthermore, for reachability probability, discrete voxel and autoencoder models are also introduced to calculate the reachability probability of the candidate target pose, and the reachability probability calculated by the MMM is weighted and summed to more accurately calculate the reachability probability of different poses and construct a probability database. By establishing a probability database for control, the "reachability probability" is explicitly introduced into the grasping / placement scoring system, avoiding failures caused by "being able to grasp but not being able to reach," thus unifying the vision-motion planning closed-loop optimization objective as "reachable grasping quality." Compared to traditional IK residuals, which only reflect single-point errors, this application simultaneously quantifies the joint feasibility of position and rotation, improving the success rate of tasks in confined spaces. The weighted fusion strategy of grasping / placement pose evaluation and reachability probability unifies the vision-motion planning closed-loop optimization objective as "reachable grasping quality," effectively improving the success rate of robotic arms in complex environments.

[0091] Figure 3 An internal structural diagram of a computer device in one embodiment is shown. This computer device can specifically be a terminal or a server. Figure 3 As shown, the device includes: a processor 310 and a memory 311 storing a computer program; wherein, Figure 3 The processor 310 shown in the diagram does not indicate that there is only one processor 310, but only indicates the positional relationship of the processor 310 relative to other devices. In practical applications, there can be one or more processors 310; similarly, Figure 3 The memory 311 illustrated herein has the same meaning, that is, it is only used to indicate the positional relationship of memory 311 relative to other devices. In practical applications, there can be one or more memories 311. When the processor 310 runs the computer program, the method applied to the above-mentioned device is implemented.

[0092] The device may also include at least one network interface 312. The various components of the device are coupled together via a bus system 313. It is understood that the bus system 313 is used to implement communication between these components. In addition to a data bus, the bus system 313 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 3 The general designated all buses as Bus System 313.

[0093] The memory 311 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 311 described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0094] The memory 311 in this embodiment of the invention is used to store various types of data to support the operation of the device. Examples of this data include: any computer programs used to operate on the device, such as operating systems and applications; contact data; phonebook data; messages; pictures; videos, etc. The operating system includes various system programs, such as the framework layer, core library layer, driver layer, etc., used to implement various basic services and handle hardware-based tasks. Applications can include various applications, such as media players, browsers, etc., used to implement various application services. Here, the program implementing the method of this embodiment of the invention can be included in the application.

[0095] Based on the same inventive concept as the foregoing embodiments, this embodiment also provides a computer-readable storage medium storing a computer program. The computer-readable storage medium can be a magnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM), etc.; it can also be various devices including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc. When the computer program stored in the computer-readable storage medium is run by a processor, it implements the above method. For the specific steps implemented when the computer program is executed by the processor, please refer to [link to relevant documentation]. Figure 1 The description of the illustrated embodiments will not be repeated here.

[0096] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0097] In this document, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, which includes not only the elements listed but also other elements not expressly listed.

[0098] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for training a reachability probability calculation model, characterized in that, Includes the following steps: Based on the design parameters of the robotic arm, a preset number of joint state combinations are randomly generated, and the end pose data to be screened for each joint state combination is determined by forward kinematics solution. Identify the unreachable end pose data in the end pose data to be screened, remove the unreachable end pose data from the end pose data to be screened to obtain reachable end pose data, and generate a training set based on the reachable end pose data. A multivariate Gaussian mixture model is trained based on the training set to fit the spatial distribution of the reachable pose data, and an reachability probability calculation model is generated based on the multivariate Gaussian mixture model. The reachability probability calculation model is used to output the reachability probability corresponding to any end pose. The step of training a multivariate Gaussian mixture model for fitting the spatial distribution of the reachable pose data based on the training set includes: training the model according to a preset training method based on the training set to obtain training parameters, the training parameters including mixture weights, mean vector, and covariance matrix; constructing the multivariate Gaussian mixture model based on the training parameters, the multivariate Gaussian mixture model taking pose data as input and processing it to obtain the log-likelihood value corresponding to the pose data, the log-likelihood value being used to describe the spatial probability distribution density of the pose data. The step of generating an reachability probability calculation model based on the multivariate Gaussian mixture model includes: inputting all samples in the training set into the multivariate Gaussian mixture model to obtain the log-likelihood values ​​of all samples; determining the 99th percentile of the log-likelihood values ​​of all samples based on the log-likelihood values ​​of all samples, and marking the 99th percentile as a reference standard value; obtaining a preset temperature coefficient; generating the reachability probability calculation model based on the temperature coefficient, the reference standard value, and the multivariate Gaussian mixture model, wherein the reachability probability calculation model takes the end pose as input, processes the end pose according to the multivariate Gaussian mixture model to obtain the log-likelihood value, and calculates the corresponding reachability probability based on the log-likelihood value, the temperature coefficient, and the reference standard value.

2. The reachability probability calculation model training method as described in claim 1, characterized in that, The step of determining the unreachable end-effector pose data in the end-effector pose data to be screened includes: Obtain the self-collision condition constraints of the robotic arm for the joint state combinations, and determine the unreachable end-effector pose data corresponding to the joint state combinations that do not satisfy the self-collision constraints from the preset number of joint state combinations as the unreachable end-effector pose data; and / or, Obtain the task workspace corresponding to the robotic arm in the current task, wherein the task workspace is the neighborhood space surrounding the target object corresponding to the current task; determine the unreachable end-effector pose data by identifying the end-effector pose data that are not in the neighborhood space; and / or, Obtain obstacle point clouds in the external environment of the robotic arm; determine the unreachable end pose data corresponding to the joint state combinations in the preset number of joint state combinations that collide with the robotic arm point cloud obtained based on the design parameters.

3. The reachability probability calculation model training method as described in claim 1, characterized in that, The step of generating a training set based on the reachable end-effector pose data includes: The reachable end-effector pose data is subjected to standard normalization to obtain the first data; An anomaly removal operation is performed on the first data using a preset filtering method to remove abnormal data from the first data, and the remaining data is marked as the second data. Samples are generated based on the second data, and all the samples are aggregated to obtain the training set.

4. A robotic arm control method, characterized in that, Includes the following steps: When a control command is received, at least one candidate target pose is calculated and determined based on the control command. The candidate target pose is input into the reachability probability calculation model for processing to obtain the reachability probability corresponding to the candidate target pose. The reachability probability calculation model is trained by the method described in any one of claims 1 to 3. The optimal target pose is selected from the candidate target poses based on the reachability probability. The robotic arm is controlled to move to the target position corresponding to the control command in the optimal target pose.

5. The robotic arm control method as described in claim 4, characterized in that, The step of inputting the candidate target pose into the reachability probability calculation model for processing to obtain the reachability probability corresponding to the candidate target pose includes: The candidate target pose is processed into a log-likelihood value according to the multivariate Gaussian model mixture model in the reachability probability calculation model, wherein the multivariate Gaussian model mixture model is trained by the method described in any one of claims 1 to 3; The reachability probability is calculated based on the log-likelihood value, a preset temperature coefficient, and a reference standard value. The temperature coefficient is a preset value, and the reference standard value is obtained by training the reachability probability calculation model.

6. The robotic arm control method as described in claim 4, characterized in that, The method further includes inputting the candidate target pose into the reachability probability calculation model for processing to obtain the reachability probability corresponding to the candidate target pose. The probability output by the reachability probability calculation model is labeled as the first reachability probability; At least one pre-built and trained auxiliary computing model is obtained, and the candidate target pose is respectively input into the at least one auxiliary computing model to obtain at least one second reachability probability output by each of the at least one auxiliary computing model; The reachability probability corresponding to the candidate target pose is determined based on the first reachability probability and at least one second reachability probability.

7. The robotic arm control method as described in claim 6, characterized in that, The at least one reachability probability calculation model includes a first auxiliary calculation model, which is used for: Based on the design parameters of the robotic arm, determine the workspace and reachable end-effector pose data of the robotic arm; The workspace is divided into multiple discrete units, each of which represents a three-dimensional space of the same preset size within the workspace. Based on the number of reachable end pose data contained in each discrete unit, the reachability probability corresponding to each discrete unit is determined; Obtain the candidate target pose in the workspace, determine the target discrete cell in which the candidate target pose is located, and determine the reachability probability corresponding to the target discrete cell as the second reachability probability corresponding to the candidate target pose.

8. The robotic arm control method as described in claim 6, characterized in that, The at least one reachability probability calculation model includes a second auxiliary calculation model, which is used for: The candidate target pose is input into the trained autoencoder model to obtain the reconstruction error value of the candidate target pose under the autoencoder model. The autoencoder model is trained based on the training set and is used to generate and parse the latent representation corresponding to the reachable end pose data. The second reachability probability output by the second auxiliary calculation model is obtained by calculating based on the reconstruction error value, the reference standard value and the preset temperature parameters. The reference standard value is the 99th percentile of the reconstruction error value corresponding to all samples in the training set.

9. A computer device, characterized in that, Including processor and memory; The processor is used to execute a computer program stored in the memory to implement the reachability probability calculation model training method as described in any one of claims 1 to 3, or the robotic arm control method as described in any one of claims 4 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the reachability probability calculation model training method as described in any one of claims 1 to 3, or the robotic arm control method as described in any one of claims 4 to 8.

Images