Adaptive grasping method and system for irregular cargo-oriented embodied intelligent robot

Abstract

The application provides a somatic intelligent robot adaptive grasping method and system for irregular cargos, and relates to the technical field of intelligent robots. The method comprises the following steps: analyzing a target image of a target grasping scene collected by a depth camera; inputting a target three-dimensional voxel into an implicit grasping reasoning model to obtain an output result, including at least one candidate grasping pose; evaluating and comparing the candidate grasping poses to obtain an optimal grasping pose, and performing grasping simulation; if the simulation result meets a predetermined execution constraint, activating an end effector of an intelligent robot to grasp and execute the target irregular cargo based on the optimal grasping pose. The application solves the technical problem that, in the prior art, due to the complex shape and stacking occlusion of irregular cargos, the grasping failure rate is high, collision or cargo damage is prone to occur, and the overall efficiency of the grasping operation is affected. The grasping pose is pre-verified, and the efficiency of the irregular cargo grasping operation is improved.

Description

Technical Field

[0001] This application relates to the field of intelligent robot technology, specifically to an adaptive grasping method and system for embodied intelligent robots for handling irregular goods. Background Technology

[0002] In logistics sorting, industrial assembly, and waste recycling, it is often necessary to handle goods with highly irregular shapes, varied surface materials, and overlapping and occlusion. Traditional robotic grasping methods typically rely on high-precision 3D reconstruction to output a grasping pose and execute it. However, irregular goods often have characteristics such as partial occlusion, complex geometry, and uneven center of gravity distribution, which makes the grasping pose obtained by a single visual inference highly uncertain in terms of quality. Direct execution is prone to grasping failure, goods slipping, or even collisions and damage with surrounding objects.

[0003] In summary, existing technologies suffer from technical problems such as high grasping failure rates, easy collisions or damage to goods due to the complex shapes of irregular goods and stacking obstructions, which further affect the overall efficiency of grasping operations. Summary of the Invention

[0004] The purpose of this application is to provide an adaptive grasping method and system for embodied intelligent robots for irregular goods, in order to solve the technical problems in the prior art that the complex shape and stacking of irregular goods lead to a high grasping failure rate, easy collision or damage to the goods, and further affect the overall efficiency of grasping operations.

[0005] To achieve the above objectives, this application provides an adaptive grasping method and system for embodied intelligent robots for irregular goods.

[0006] In a first aspect, this application provides an adaptive grasping method for embodied intelligent robots targeting irregular goods. This method is implemented through an adaptive grasping system for embodied intelligent robots targeting irregular goods. The method includes: analyzing target images of a target grasping scene captured by a depth camera to obtain target three-dimensional voxels, where the target grasping scene refers to a grasping operation scene containing target irregular goods; inputting the target three-dimensional voxels into an implicit grasping inference model to obtain the output result of the implicit grasping inference model, where the output result includes at least one candidate grasping pose; evaluating and comparing the candidate grasping poses to obtain an optimal grasping pose, and performing a grasping simulation on the optimal grasping pose to obtain a simulation result; if the simulation result satisfies predetermined execution constraints, activating the end effector of the intelligent robot to perform grasping execution on the target irregular goods based on the optimal grasping pose.

[0007] Optionally, the first image and the second image in the target image are extracted sequentially; an arbitrary voxel in the target grasping scene is obtained, and a first distance value of the arbitrary voxel in the first image and a second distance value in the second image are obtained; the average of the first distance value and the second distance value is taken as the arbitrary fusion distance value corresponding to the arbitrary voxel; the target three-dimensional voxel is established based on the correspondence between the arbitrary voxel and the arbitrary fusion distance value; wherein, the first image is a depth image obtained by sampling the first point of N points with a hemispherical trajectory at a predetermined height position in the target grasping scene as the shooting point of the depth camera, and the second image is a depth image obtained by sampling the second point of N points with a hemispherical trajectory at a predetermined height position in the target grasping scene as the shooting point of the depth camera, where N is an integer greater than or equal to 6; wherein, the arbitrary voxel corresponds to an arbitrary cubic region in the target grasping scene, and the side length of the arbitrary cubic region is 4.5 mm.

[0008] Optionally, using the first image as a constraint, obtain an arbitrary distance value from the arbitrary voxel to the optical center of the depth camera; using the first image as a constraint, obtain an arbitrary angle of incidence between the arbitrary voxel and the acquisition line of sight of the depth camera; adjust the arbitrary angle of incidence by retrieving a preset weight factor and normalize it to obtain an arbitrary weight; calculate the first distance value based on the arbitrary weight and the arbitrary distance value.

[0009] Optionally, the implicit grasping inference model includes a shared feature encoder, a first decoder, and a second decoder, wherein the first decoder and the second decoder are parallel components. The target 3D voxel is input into the implicit grasping inference model to obtain its output, which includes: acquiring any grasping point in the target 3D voxel; decoding the grasping pose of the arbitrary grasping point using the first decoder based on the shared feature encoder to obtain an arbitrary grasping quality index; simultaneously, decoding the 3D shape of the arbitrary grasping point using the second decoder based on the shared feature encoder to obtain an arbitrary space occupancy probability; constructing arbitrary grasping prediction information for the arbitrary grasping point based on the arbitrary grasping quality index and the arbitrary space occupancy probability; and adding the arbitrary grasping pose corresponding to the arbitrary grasping point to the output when the arbitrary grasping prediction information meets predetermined grasping conditions.

[0010] Optionally, the first decoder decodes the arbitrary grasping pose based on the shared feature encoder to obtain an arbitrary grasping pose; it introduces a grasping quality evaluation index to perform feature analysis on the arbitrary grasping pose to obtain grasping quality parameters; and it performs label-based normalized weighted processing on the grasping quality parameters to obtain the arbitrary grasping quality index; wherein, the grasping quality evaluation index includes a grasping quality score, a grasping opening width, and a grasping direction.

[0011] Optionally, the predetermined grasping conditions include hard conditions and flexible conditions, wherein the hard conditions refer to the predetermined opening width of the end effector, and the flexible conditions include a predetermined grasping quality index threshold and a predetermined space occupancy probability threshold.

[0012] Optionally, a target digital twin is constructed; the real-time pose of the end effector is obtained, and the grasping path is determined by combining the optimal grasping pose; the grasping path is simulated using the target digital twin to obtain the simulation result.

[0013] Optionally, if the simulation result does not meet the predetermined execution constraint, the grasping path is adjusted to obtain an adjusted path, and the adjusted path is simulated; wherein, the predetermined execution constraint refers to the simulated grasping process without collision and successfully grasping the target irregular goods.

[0014] Optionally, the end effector includes a first fine-grained adjustment component, a second fine-grained adjustment component, and a flexible gripper, wherein the first fine-grained adjustment component is used to control the flexible gripper to reach the target irregular cargo at a first speed, the second fine-grained adjustment component is used to control the flexible gripper to reach the target irregular cargo at a second speed, and the flexible gripper is equipped with a pressure sensor for adaptively conforming to the target irregular cargo.

[0015] Secondly, this application also provides an adaptive grasping system for embodied intelligent robots for irregular goods, used to execute the adaptive grasping method for embodied intelligent robots for irregular goods as described in the first aspect. The adaptive grasping system for embodied intelligent robots for irregular goods includes: a target grasping scene analysis module, used to analyze target images of a target grasping scene captured by a depth camera to obtain target three-dimensional voxels, wherein the target grasping scene refers to a grasping operation scene containing target irregular goods; a target three-dimensional voxel construction module, used to input the target three-dimensional voxels into an implicit grasping inference model to obtain the output result of the implicit grasping inference model, wherein the output result includes at least one candidate grasping pose; a grasping pose evaluation and simulation module, used to evaluate and compare the candidate grasping poses to obtain an optimal grasping pose, and perform grasping simulation on the optimal grasping pose to obtain a simulation result; and a grasping execution module, used to activate the end effector of the intelligent robot to perform grasping execution on the target irregular goods based on the optimal grasping pose if the simulation result satisfies predetermined execution constraints.

[0016] One or more technical solutions provided in this application have at least the following technical effects or advantages: By analyzing the target image of the target grasping scene captured by the depth camera, a three-dimensional voxel of the target is obtained. The target grasping scene refers to a grasping operation scenario containing irregularly shaped goods. The three-dimensional voxel of the target is input into an implicit grasping inference model, and the output result of the implicit grasping inference model is obtained. The output result includes at least one candidate grasping pose. The candidate grasping poses are evaluated and compared to obtain the optimal grasping pose, and a grasping simulation is performed on the optimal grasping pose to obtain a simulation result. If the simulation result meets predetermined execution constraints, the end effector of the intelligent robot is activated to grasp the irregularly shaped goods based on the optimal grasping pose. In other words, multiple candidate grasping poses are quickly generated from three-dimensional voxels through the implicit grasping inference model. After evaluating and obtaining the optimal pose, a grasping simulation is performed. Only when the simulation result meets predetermined execution constraints can the end effector be activated. This effectively filters out infeasible grasping poses before physical execution, reduces the risk of collisions and grasping failures, and improves the success rate and operational safety of grasping irregularly shaped goods.

[0017] The above description is merely an overview of the technical solution of this application. To better understand the technical means of this application and to facilitate its implementation according to the description, and to make the above and other objects, features, and advantages of this application more apparent, specific embodiments of this application are described below. It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent through the following description. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the adaptive grasping method of the embodied intelligent robot for irregular goods, as described in this application.

[0020] Figure 2 This is a schematic diagram of the adaptive grasping system of the embodied intelligent robot for irregular goods, as described in this application.

[0021] Figure labeling: Target grasping scene analysis module 11, Target 3D voxel construction module 12, Grasping pose evaluation simulation module 13, Grasping execution module 14. Detailed Implementation

[0022] This application provides an adaptive grasping method and system for embodied intelligent robots for irregular goods, solving the technical problems in existing technologies where the complex shape and stacking of irregular goods lead to high grasping failure rates, collisions, or damage, further affecting the overall efficiency of grasping operations. By using an implicit grasping inference model to rapidly generate multiple candidate grasping poses from 3D voxels, and evaluating the optimal pose, grasping simulation is performed. Only when the simulation results meet predetermined execution constraints can the end effector be activated. This effectively filters out infeasible grasping poses before physical execution, reducing the risk of collisions and grasping failures, and improving the success rate and operational safety of grasping irregular goods.

[0023] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. It should be understood that this application is not limited to the exemplary embodiments described herein. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application. It should also be noted that, for ease of description, only the parts related to this application are shown in the accompanying drawings, not all of them.

[0024] Example 1, please refer to the appendix. Figure 1This application provides an adaptive grasping method for embodied intelligent robots for irregular goods. The method is applied to an adaptive grasping system for embodied intelligent robots for irregular goods, and specifically includes the following steps: Analyze the target image of the target grasping scene captured by the depth camera to obtain the target three-dimensional voxel, wherein the target grasping scene refers to the grasping operation scene containing the target irregular goods.

[0025] Furthermore, this application also includes the following steps: sequentially extracting a first image and a second image from the target image; obtaining an arbitrary voxel in the target grasping scene, and obtaining a first distance value of the arbitrary voxel in the first image and a second distance value in the second image; taking the average of the first distance value and the second distance value as an arbitrary fusion distance value corresponding to the arbitrary voxel; establishing the target three-dimensional voxel based on the correspondence between the arbitrary voxel and the arbitrary fusion distance value; wherein, the first image is a depth image obtained by sampling N points with a hemispherical trajectory at a predetermined height position in the target grasping scene as the first point of the depth camera, and the second image is a depth image obtained by sampling N points with a hemispherical trajectory at a predetermined height position in the target grasping scene as the second point of the depth camera, where N is an integer greater than or equal to 6; wherein, the arbitrary voxel corresponds to an arbitrary cubic region in the target grasping scene, and the side length of the arbitrary cubic region is 4.5 mm.

[0026] Furthermore, this application also includes the following steps: using the first image as a constraint, obtaining an arbitrary distance value from the arbitrary voxel to the optical center of the depth camera; using the first image as a constraint, obtaining an arbitrary angle of incidence between the arbitrary voxel and the acquisition line-of-sight direction of the depth camera; adjusting the arbitrary angle of incidence by retrieving a preset weighting factor and normalizing it to obtain an arbitrary weight; and calculating the first distance value based on the arbitrary weight and the arbitrary distance value.

[0027] Specifically, a depth camera is fixedly mounted on the end flange or external bracket of the embodied intelligent robot, and the transformation matrix between the camera coordinate system and the embodied intelligent robot's base coordinate system is determined through hand-eye calibration. A depth camera is an imaging device capable of simultaneously acquiring the distance from the object surface to the camera's optical center for each pixel in the target grasping scene, such as cameras based on structured light, time-of-flight, or binocular stereo vision principles. Its output is a depth image, with each pixel value representing a distance. The target grasping scene is the physical space area where grasping operations are performed in actual operations. It contains at least one irregularly shaped target cargo, such as irregularly shaped castings, deformed parts, or scattered fruits and vegetables, and may also include environmental objects such as material bins, conveyor belts, and other goods.

[0028] The embodied intelligent robot is controlled to move the camera to the first pre-planned shooting position, typically located above or to the side of the target grasping scene, ensuring that the main area of ​​the target grasping scene is within the camera's field of view. The camera is then triggered to capture one frame of data, obtaining a depth image with a resolution of width x height pixels. Each pixel records the distance from the camera's optical center to its corresponding scene point. Similarly, the camera is moved to a second shooting position, and the acquisition process is repeated to obtain a second depth image. These two images together constitute the target image set.

[0029] Before acquiring multi-view depth images, a virtual hemisphere is constructed at a certain height directly above the center of the target capture scene. The center of the hemisphere is located at the center of the scene or the center of the key area, and the radius is determined according to the camera's field of view and the required viewing angle. N points are sampled uniformly or strategically on this hemisphere. Each point serves as a shooting position for the depth camera and typically includes the zenith, several points on the horizontal circumference, and points at the intermediate pitch angle.

[0030] Using the center point of the target capture scene as the center of a hemisphere, and setting the hemisphere radius R, typically between 0.3 and 0.8 meters, ensures the camera can clearly see the entire scene from different angles. N points are uniformly sampled on the hemisphere surface, where N is a positive integer and not less than 6, with the zenith direction considered as a point. Several rings with varying elevation angles are taken within multiple azimuth planes, and points are taken at equal intervals on each ring to generate the 3D coordinates and attitude of each point. The attitude is designed so that the camera's line of sight always points towards the center of the sphere, i.e., the center of the scene, with each viewpoint focusing on the scene center. After trajectory planning, the robotic arm moves sequentially to each point, capturing a depth image at each point. After acquisition, all images are stored in an image list, recording the point number corresponding to each image. Following the trajectory order, the image of the first point is extracted from the image list and labeled as the first image; the first point is the first sampling point in the predefined trajectory order. The image of the second point is extracted and labeled as the second image; the second point is the second sampling point in the trajectory order.

[0031] Determine the bounding box extent of the target grasping scene in 3D space. Each voxel corresponds to an arbitrary cubic region within the target grasping scene, and the side length of each cubic region is 4.5 mm. All voxels have equal side lengths, ensuring consistency in spatial discretization. For example, using the robot's base coordinate system as a reference, set the minimum and maximum values ​​in the X, Y, and Z directions. The scene bounding box should completely cover the area that the bin, goods, and robotic arm end effector might reach. Based on the preset voxel side lengths, such as 4.5 mm, calculate the number of voxels in the X, Y, and Z directions respectively: Voxel count in each direction = (maximum value in that direction - minimum value in that direction) / 0.0045. If the calculation result is not an integer, round it up to slightly expand the bounding box. Create a 3D array with dimensions equal to the number of voxels in the three directions to store the distance value corresponding to each voxel. Calculate the center coordinates of the current voxel based on its index in the length, width, and height directions. For the current index, calculate the 3D coordinates of the voxel center. Multiply the index by the voxel side length to obtain the offset from the box boundary to the voxel center. Add the minimum value of the box boundary to obtain the coordinates of the voxel center in real space. These voxel center coordinates represent the spatial location of any voxel being processed.

[0032] Based on the spatial coordinates of the current voxel center and the spatial coordinates of the camera optical center when the first image was captured, the straight-line distance between the two points is calculated. The differences between the voxel center and the optical center in the length, width, and height directions are calculated respectively. The squares of each difference are added together to obtain the sum of squares. The square root of this sum of squares is then taken to obtain the straight-line distance from the voxel to the optical center.

[0033] Based on the principal optical axis of the first camera, the vector from the voxel center to the optical center is transformed into the camera's own coordinate system. In the camera coordinate system, the principal optical axis is the depth axis. The depth coordinates of the voxel center in the camera coordinate system are obtained. This depth coordinate is divided by the previously calculated straight-line distance from the voxel to the optical center, yielding a ratio equal to the cosine of the angle of incidence. The inverse cosine function is used to calculate this ratio, yielding the angle of incidence in degrees. A smaller angle of incidence indicates that the voxel is closer to the image center, resulting in higher depth measurement accuracy; a larger angle of incidence indicates that the voxel is closer to the image edge, leading to greater measurement error.

[0034] The system retrieves a pre-stored weight mapping table or calculation formula to obtain an initial weight value. For example, the initial weight is set to 1.0 when the incident angle is less than 10 degrees, linearly decreasing from 1.0 to 0.5 when the incident angle is between 10 and 45 degrees, and set to 0.3 when the incident angle is greater than 45 degrees. A smoother mapping, such as a Gaussian function, is used. After obtaining the initial weights, normalization is required. The average initial weights of all voxels across the entire viewpoint corresponding to the first image are calculated, and then the initial weight of each voxel is divided by this average to obtain the normalized weights.

[0035] Multiplying the normalized arbitrary weights by an arbitrary distance value yields the first distance value, reflecting the effective distance after incident angle correction from the first viewpoint. Similarly, repeating the aforementioned steps yields the second distance value. Specifically, for the same voxel, the optical center and principal optical axis direction of the camera corresponding to the second image are now used. The straight-line distance from the voxel center to the second optical center is calculated, and the incident angle of the voxel center relative to the principal optical axis of the second camera is calculated. Using the same preset weight factor, the original weights are obtained based on the incident angle, and then the average of the original weights of all voxels in the second viewpoint is normalized to obtain the normalized weights under the second viewpoint. Multiplying this normalized weight by the second original straight-line distance yields the second distance value.

[0036] The average of the first and second distance values ​​is calculated, and the result is the fused distance value for that voxel. Since the weights of the two viewpoints have been normalized, and the normalization method ensures the correct dimensions of the average result, the fused distance value is equivalent to the effective distance from the center of the voxel to the surface of an object in the scene, taking into account the measurement reliability of both viewpoints. If the voxel is located inside an object, the distance value is negative; if it is located outside, it is positive. After calculation, this fused distance value is stored in the array element corresponding to that voxel.

[0037] After traversing all voxels and calculating the blending distance value for each voxel, the spatial coordinates of each voxel are associated with its blending distance value and stored in a 3D array. The three dimensions correspond to the number of voxels in the length, width, and height directions, respectively. For example, the array element position (i, j, k) stores the blending distance value of the voxel with indices i, j, and k. The target 3D voxel is a 3D mesh, and each mesh point stores a floating-point number representing the signed distance from that point to the scene surface.

[0038] For example, the target's center in the scene is located at (0.000m, 0.000m, 0.250m). A hemispherical trajectory with a radius of 0.6m is planned. Sampling point 1 (zenith) has coordinates of (0.000m, 0.000m, 0.850m) and an orientation vertically downwards; point 2 has an azimuth of 0° and a pitch of 45° with coordinates of (0.424m, 0.000m, 0.674m) and an orientation pointing towards the scene center. Two depth images are acquired sequentially, with a resolution of 1280×720 pixels and a depth scale factor of 0.001m / unit. The length of the scene bounding box ranges from -0.450m to 0.450m, the width from -0.400m to 0.400m, and the height from 0.000m to 0.500m. The voxel side length is 4.5mm. The model contains 200 voxels along its length, 178 along its width, and 112 along its height, for a total of 3,987,200 voxels. The voxel at index (100, 89, 55) has center coordinates of (0.00225m, 0.00275m, 0.24975m). The first distance is calculated to be 0.6320m, and the second distance is 0.000190m, resulting in a fusion distance of 0.316095m. This value is stored in the

[100]

[89]

[55] position of the 3D array. This calculation is repeated for all 3,987,200 voxels to obtain the complete target 3D voxel model, with a total data size of approximately 15.2MB.

[0039] By sequentially extracting depth images from two different perspectives and calculating the weighted distance value for each voxel and then taking the average, a target 3D voxel model fused with multi-view information was constructed, which significantly improved the quality of monocular depth data before multi-view fusion.

[0040] The target 3D voxel is input into the implicit grasping inference model to obtain the output result of the implicit grasping inference model, wherein the output result includes at least one candidate grasping pose.

[0041] Furthermore, this application also includes the following steps: the implicit grasping inference model includes a shared feature encoder, a first decoder, and a second decoder, wherein the first decoder and the second decoder are parallel components. The target three-dimensional voxel is input into the implicit grasping inference model to obtain the output result of the implicit grasping inference model, including: obtaining any grasping point in the target three-dimensional voxel; decoding the grasping pose of the arbitrary grasping point based on the shared feature encoder using the first decoder to obtain an arbitrary grasping quality index; simultaneously, decoding the three-dimensional shape of the arbitrary grasping point based on the shared feature encoder using the second decoder to obtain an arbitrary space occupancy probability; forming arbitrary grasping prediction information for the arbitrary grasping point based on the arbitrary grasping quality index and the arbitrary space occupancy probability; when the arbitrary grasping prediction information meets a predetermined grasping condition, adding the arbitrary grasping pose corresponding to the arbitrary grasping point to the output result.

[0042] Furthermore, this application also includes the following steps: the first decoder decodes the arbitrary grasping point based on the shared feature encoder to obtain an arbitrary grasping pose; a grasping quality evaluation index is introduced to perform feature analysis on the arbitrary grasping pose to obtain grasping quality parameters; the grasping quality parameters are subjected to label-based normalized weighted processing to obtain the arbitrary grasping quality index; wherein, the grasping quality evaluation index includes a grasping quality score, a grasping opening width, and a grasping direction.

[0043] Furthermore, this application also includes the following steps: the predetermined grasping conditions include hard conditions and flexible conditions, wherein the hard conditions refer to the predetermined opening width of the end effector, and the flexible conditions include a predetermined grasping quality index threshold and a predetermined space occupancy probability threshold.

[0044] Specifically, the target 3D voxels are input into a pre-trained implicit grasping inference model, which includes a shared feature encoder and two parallel-connected back-end decoders. The implicit grasping inference model is a deep learning neural network model whose core feature is that it does not explicitly reconstruct the complete 3D mesh or point cloud of the object, but instead learns an implicit function directly from the 3D voxel data. This function can directly output the grasping feasibility and local shape attributes of any 3D coordinate point in space when that point is used as the grasping center. This implicit representation is particularly suitable for irregularly shaped goods that exhibit self-occlusion or stacking. The shared feature encoder is the front-end backbone network of the model, responsible for multi-level feature extraction from the input 3D voxels. The shared feature encoder consists of multiple stacked 3D convolutional layers. Each layer scans the voxel space through a convolutional kernel, progressively expanding the receptive field and extracting higher-semantic feature maps. The output of the shared feature encoder is a feature map cube with reduced resolution but increased channel count, containing the geometry and contextual information of the entire scene. The final layer of the encoder outputs a feature map with a spatial resolution 1 / 8 that of the original voxels, but the feature vector at each location reaches 256 dimensions, ensuring that the subsequent decoder can perceive both local details and utilize global context. The first decoder, relative to the grasping pose decoder, is a lightweight neural network branch connected in parallel with the shared feature encoder. Its input is not the entire feature map, but rather the feature vector corresponding to any grasping point location in the encoder's output feature map. The first decoder's task is, given a 3D spatial point, to output the grasping pose parameters required to perform grasping at that point, as well as a preliminary grasping quality score. In the decoder's output layer, the grasping direction is not directly regressed using quaternions, but rather uses two angles in spherical coordinates. The second decoder, equivalent to a shape decoder, is another parallel branch, sharing the exact same encoder features as the first decoder. The second decoder's input is also the feature vector corresponding to the same grasping point, and its output is a spatial occupancy probability—the proportion of the volume occupied by the cargo entity within a cube centered at that grasping point and with sides equal to the predicted grasping opening width. Because the object has an irregular shape, the closed area of ​​the gripper may be partially occupied by the object and partially empty. The space occupancy probability can help the model avoid selecting gripping points where the gripper will collide with the object's protrusions or fall into the void.

[0045] The first and second decoders are computationally parallel, simultaneously extracting feature vectors corresponding to the grasping points from the output of the shared feature encoder, and simultaneously calculating the grasping pose and occupancy probability. This not only shortens the inference time, but more importantly, it allows the two tasks to constrain each other during training. The shared encoder must learn features that are beneficial to both grasping prediction and shape occupancy, forcing the model to understand the geometric nature of irregular goods more deeply, rather than overfitting to a single task.

[0046] The internal modules and hierarchical connections of the model are as follows: The shared feature encoder consists of six 3D convolutional blocks, each containing: a 3D convolutional layer, a batch normalization layer, a ReLU activation layer, and a max pooling layer. The first convolutional block has a kernel size of 5×5×5, a stride of 2, and 32 output channels; the second convolutional block has a kernel size of 3×3×3, a stride of 2, and 64 output channels; the third convolutional block has a kernel size of 3×3×3, a stride of 2, and 128 output channels; the fourth convolutional block has a kernel size of 3×3×3, a stride of 2, and 256 output channels; the fifth convolutional block has a kernel size of 3×3×3, a stride of 1, and 256 output channels; the sixth convolutional block has a kernel size of 1×1×1, used for dimensionality reduction, and 128 output channels. After processing through these six blocks, the original voxel spatial resolution is reduced from 200×178×112 to 25×23×14, with each pooling iteration halving the resolution, resulting in 128 feature channels. The feature map output from the last layer of the encoder is a five-dimensional tensor (channel × height × width × depth), where each voxel coordinate corresponds to a 128-dimensional feature vector, encoding the geometric information of a large neighborhood centered on that voxel. For any grasping point, a 128-dimensional feature vector at that point is extracted from the encoder feature map using trilinear interpolation. This feature vector is then input into a network consisting of three fully connected layers. The first layer maps the 128-dimensional vector to 256-dimensional, the second to 128-dimensional, and the third to 6-dimensional, including the grasping point's position offset, the two angles of the grasping direction, and the grasping opening width. An additional scalar is extracted from the penultimate layer of the first decoder, compressed to between 0 and 1 using a sigmoid function, and used as the grasping quality score. The second decoder also extracts the same 128-dimensional feature vector from the encoder feature map and inputs it into a network consisting of two fully connected layers: the first layer maps to 64-dimensional, and the second to 1-dimensional. The output is then processed by a sigmoid function to obtain a spatial occupancy probability between 0 and 1.

[0047] An irregular cargo grasping environment was constructed. 1000 irregular shapes were randomly generated, including hyperquadratic surface deformation, random convex hull offset, and mesh models extracted from real scan data. Each shape was randomly placed in the scene, and a heuristic-based grasping sampler generated 30-50 positive grasping poses and 30-50 negative poses. The spatial occupancy probability of each grasping point was recorded by projecting a certain number of rays along the grasping direction in the simulation and calculating the proportion of objects hit. Approximately 80,000 training samples were generated in total. The PyTorch framework was used, with the Adam optimizer, an initial learning rate of 0.001, decaying to 0.9 every 10 epochs, for a total of 50 training epochs. The batch size was set to 8. During training, the input to each training sample was a 128×128×64 normalized voxel block, and the output was the 6D pose parameters, quality score, and occupancy probability of the corresponding center grasping point. For each training sample, a grab point within the positive-negative balance range is randomly selected instead of all points to avoid class imbalance.

[0048] The shared feature encoder design causes changes in the distance values ​​within voxels, such as the distance values ​​of voxels near the object's surface transitioning from positive to negative, to directly alter the pattern of the feature vector, thus simultaneously affecting both grasp pose prediction and occupancy probability. For example, when a grasp point is close to the object's surface, a significant gradient change occurs in the feature vector. The first decoder tends to output a pose with a smaller opening width and a direction pointing towards the surface normal, while the second decoder outputs a high occupancy probability close to 1. Conversely, if the grasp point is in free space, the first decoder may output a larger opening width, and the second decoder may output an occupancy probability close to 0.

[0049] The shared feature encoder performs multiple 3D convolutions and downsampling on the input 3D voxels to extract feature maps with rich spatial context information. It traverses all candidate grasping points in the voxel space. For each grasping point being processed, two branches are executed in parallel. In the first decoder branch, based on the local features surrounding the point, the grasping pose parameters required for grasping at that point are decoded, including the 3D coordinate offset of the grasping point, the opening width of the gripper, and the grasping direction. A set of grasping quality evaluation metrics is introduced, including the grasping quality score, opening width, and grasping direction directly output by the model. Feature analysis is performed on these parameters, such as calculating the ratio of the opening width to the maximum opening of the end effector and evaluating the consistency between the grasping direction and the object surface normal. These analysis results are labeled, normalized, and weighted, such as assigning different weights to different metrics (quality score weight 0.6, opening width weight 0.2, and direction weight 0.2). The weighted items are then summed to obtain a scalar value, namely the grasping quality index. The crawl quality parameter is the original value output by the model, while the crawl quality index is the result obtained after label normalization and weighting. The higher the crawl quality index, the more reliable the crawl point is.

[0050] Simultaneously, in the second decoder branch, based on the same shared features, a 3D shape decoding is performed on the grasping point, outputting a space occupancy probability. This probability represents the proportion of the cube space centered on the grasping point and with a side length equal to the predicted opening width, occupied by the object. The space occupancy probability is a value between 0 and 1 output by the second decoder, indicating the proportion of the cube space centered on the grasping point and with a side length equal to the grasping opening width, occupied by the object. A probability close to 0 indicates the object is located in free space, while a probability close to 1 indicates the object is located inside an irregular object.

[0051] The grasping quality index and space occupancy probability are combined to form the prediction information for the grasping point. This prediction information is checked to see if it meets predetermined grasping conditions. For hard conditions, the predicted opening width is determined to be less than or equal to the actual maximum opening width of the robot's end effector. If satisfied, flexible conditions are further determined: the grasping quality index is greater than a preset threshold (e.g., 0.75), and the space occupancy probability is less than a preset threshold (e.g., 0.15). Only when both hard and flexible conditions are met simultaneously will the grasping pose corresponding to the grasping point be added to the candidate grasping pose list. This method eliminates the need for explicit reconstruction of the complete 3D model of the object, learning grasping feasibility directly from voxels, making it particularly suitable for goods with irregular shapes and severe local occlusion. Through weighted fusion of the grasping quality score, opening width, and grasping direction, the resulting grasping quality index more comprehensively reflects grasping feasibility than a single quality score. The occupancy probability provided by the second decoder effectively eliminates poses that appear good but actually cause interference between the gripper and nearby objects, reducing the risk of collisions during the grasping process.

[0052] The candidate grasping poses are evaluated and compared to obtain the optimal grasping pose, and the optimal grasping pose is used to perform a grasping simulation to obtain the simulation results.

[0053] Furthermore, this application also includes the following steps: constructing a target digital twin; obtaining the real-time pose of the end effector and determining the grasping path in combination with the optimal grasping pose; and simulating the grasping path using the target digital twin to obtain the simulation result.

[0054] Specifically, a digital twin of the target grasping scenario is constructed, including a complete kinematic model of the robot, a geometric model of the end effector, a 3D geometric model of the target grasping scenario, and precise models of environmental objects such as the bin and support. Based on the target's 3D voxel data, a triangular mesh surface model of the irregular cargo is extracted using a moving cube algorithm, and its physical properties, such as mass, center of mass position, coefficient of friction, and coefficient of restitution, are assigned. The robot's kinematic model is loaded, including the geometric dimensions of each link, joint type, joint limits, and a detailed mesh model of the end effector. Environmental models, such as the inner wall dimensions of the bin and the surface of the conveyor belt, are imported. All models are assembled into the same virtual scene, and global physical parameters such as gravitational acceleration and ground friction coefficient are set to complete the initialization of the digital twin.

[0055] After obtaining the candidate grasping pose list, a comprehensive evaluation and comparison of the candidate poses is performed to select the optimal grasping pose. For each candidate pose, the total joint motion required to move from the current end effector real-time pose to that candidate pose, as well as the margin of each joint of the robotic arm from the soft limit boundary, are calculated. The grasping quality index, motion stroke, and joint margin are weighted and comprehensively scored, such as a quality index weight of 0.7, a stroke normalization weight of 0.2, and a joint margin weight of 0.1. The pose with the highest comprehensive score is selected as the optimal grasping pose. For example, suppose there are 34 candidate grasping poses, each with a grasping quality index. The top five indices are 0.932, 0.928, 0.915, 0.902, and 0.887, respectively. The current end effector real-time pose is position (0.450m, 0.000m, 0.650m) and attitude (roll 0°, pitch 45°, yaw 0°). For the pose with an index of 0.932, the total joint range of motion is calculated to be 15.2°, and the minimum joint margin is 8.3°. For the pose with an index of 0.928, the total range of motion is 8.7°, and the minimum joint margin is 12.1°. After weighted comprehensive scoring, the pose with an index of 0.928 has a comprehensive score of 0.8231, which is higher than the 0.8152 of the pose with an index of 0.932. Therefore, it is selected as the optimal grasping pose, with the following parameters: grasping point (0.315m, 0.194m, 0.102m), grasping direction pitch 12°, azimuth -18°, and opening width 76.2mm.

[0056] The robot controller interface reads the end effector's pose in the current physical space in real time, including position coordinates and attitude angles. Using the current pose as the starting point and the optimal grasping pose as the ending point, a collision-free, smooth grasping path is generated in joint space or Cartesian space using a motion planning library. Inverse kinematics is used to convert the optimal grasping pose into target angles for each joint; spline interpolation or trapezoidal velocity curves are used to generate intermediate sequences from the current joint angle to the target joint angle; collision detection is performed at each path point in the sequence, and if a collision occurs, the trajectory is adjusted or replanned, such as by adding obstacle avoidance intermediate points.

[0057] The grasping path is input into the physical simulation engine of the digital twin and executed step by step at fixed time steps. At each time step, the physical simulation engine updates the robot's forward kinematics, the position and orientation of the gripper, and performs precise collision detection. When the gripper moves to the optimal grasping pose, it closes according to the opening width specified in that pose, and calculates the contact force and friction force between the inner surface of the gripper and the cargo mesh. Based on force closure and friction torque analysis, it is determined whether the cargo is stably grasped. After the simulation is completed, the simulation results are output. If there are no rigid body collisions during the entire movement process, i.e., the minimum distance is greater than the preset safety margin, such as 0.5mm, and the cargo does not slip when subjected to small disturbances after the gripper closes, and all joint torques do not exceed the soft limit, then the simulation result satisfies the predetermined execution constraints; otherwise, it does not satisfy the constraints, and the reason for failure is recorded, such as a collision with the bin wall at the 20th point of the path or the cargo rotating and slipping around the gripper axis after grasping.

[0058] By introducing auxiliary indicators such as motion range and joint margin, the problem of selecting a pose that may be good in terms of grasping quality index but time-consuming or close to joint limit may be avoided by simply relying on the grasping quality index. Executing the complete grasping path in the virtual environment and performing collision detection with static and dynamic obstacles can identify interference problems that may occur in actual execution. The simulation results not only provide Boolean values ​​for success or failure, but also provide detailed diagnostic information, based on which the grasping strategy is automatically adjusted to ensure that the selected pose conforms to physical laws and has high stability, which significantly improves the generation efficiency and success rate of grasping poses in complex stacked scenarios.

[0059] Furthermore, this application also includes the following steps: if the simulation result does not meet the predetermined execution constraint, the grasping path is adjusted to obtain an adjusted path, and the adjusted path is simulated; wherein, the predetermined execution constraint refers to the simulated grasping process without collision and successfully grasping the target irregular goods.

[0060] Specifically, after simulating the grasping path in the digital twin, the success flag in the simulation results is checked. If the success flag is true, the process directly proceeds to the end effector execution stage. If the success flag is false, it means the simulation results do not meet the predetermined execution constraints, indicating a collision or grasping failure. The predetermined execution constraints are pre-set standards for determining whether the grasping path can be safely executed, including simulating a collision-free grasping process and successfully grasping the target irregular cargo. A collision-free simulated grasping process means that the minimum distance between all links of the robotic arm and the end effector and any object in the scene during the entire movement must be greater than a preset safety threshold, such as 0.5 mm; successfully grasping the target irregular cargo means that after the claw closes, contact force analysis and disturbance testing determine that the cargo is stably constrained and will not slip, rotate, or fall during subsequent lifting or movement.

[0061] Based on the failure reasons in the simulation results, the corresponding path adjustment strategy is invoked. If the failure is due to a collision, the path point number and collision location of the collision are first recorded. An obstacle avoidance intermediate point is inserted near the collision point, such as between the collision point and the path point before the collision. A collision-free offset position is calculated, and the corresponding joint angles are solved using inverse kinematics. The original path is then bisected at the collision point, a new intermediate point is inserted, and a smooth curve is regenerated using spline interpolation. Simultaneously, the maximum speed of this path segment can be appropriately reduced to minimize trajectory deviation due to inertia.

[0062] If the gripping fails, i.e., slippage occurs, the optimal gripping pose parameters are fine-tuned based on contact force analysis and the slippage direction. For example, if the goods slip to the left under horizontal disturbance, the gripping point is shifted 2mm to the right, and the gripper closing force is increased by 10%. If the gripping pose is being adjusted, inverse kinematics needs to be called again to convert the new gripping pose into joint target angles. Then, the path from the current real-time pose to the new target pose is replanned. The gripping pose sequence is adjusted only if the adjustment range is limited, such as an offset of less than 5mm or an angle change of less than 5°. If the adjustment range is too large, the current pose is abandoned, and the candidate list is returned to select the second-best pose to start again.

[0063] The adjusted path is simulated again in the target digital twin until a path that meets the predetermined execution constraints is found, which is then output as the final feasible grasping path. If it still fails after reaching the maximum number of attempts, the current optimal grasping pose is reported as unavailable, and the system automatically switches to the next optimal pose in the candidate grasping pose list, restarting path planning. By setting predetermined execution constraints and automatically adjusting and re-verifying the path when simulation fails, a secure firewall is effectively built from virtual planning to physical execution, avoiding blind random modifications.

[0064] If the simulation results satisfy the predetermined execution constraints, the end effector of the intelligent robot is activated to grasp the target irregular goods based on the optimal grasping pose.

[0065] Furthermore, this application also includes the following steps: the end effector includes a first fine-grained adjustment component, a second fine-grained adjustment component, and a flexible gripper, wherein the first fine-grained adjustment component is used to control the flexible gripper to reach the target irregular cargo at a first speed, the second fine-grained adjustment component is used to control the flexible gripper to reach the target irregular cargo at a second speed, and the flexible gripper is equipped with a pressure sensor for adaptively conforming to the target irregular cargo.

[0066] Specifically, after the simulated grasping path in the digital twin satisfies predetermined execution constraints, the end effector of the embodied intelligent robot is activated. The optimal grasping path generated during the simulation is sent to the servo drivers of each joint of the embodied intelligent robot's arm, and the arm moves from the current real-time pose to the optimal grasping pose according to this path. Throughout the movement, the controller monitors the position, velocity, and torque of the joints in real time, maintaining consistency with the kinematic parameters from the simulation.

[0067] When the robotic arm reaches the optimal gripping posture, meaning the gripper is at the target gripping point and the opening width of the gripper blades is greater than the estimated thickness of the target cargo, the end effector begins the gripping operation. The end effector's control system includes two independent fine-grained adjustment components and a flexible gripper. Activating the first fine-grained adjustment component drives the gripper blades to close rapidly at a preset first speed, such as 150 mm / s. At this time, the flexible gripper has not yet contacted the cargo, and the pressure sensor reading is close to zero. The purpose of rapid closing is to reduce idle travel time and improve gripping efficiency.

[0068] When the distance between the gripper blades and the cargo surface decreases to a preset threshold, such as 10mm, or when any pressure sensor detects that the contact force has reached a small trigger value, the system immediately switches to the second fine-grained adjustment component. The second component continues to drive the gripper blades to close at a lower speed, such as 15mm / s. During this slow closing process, the elastic material of the flexible gripper begins to deform and contact the irregular cargo surface, and the pressure sensor readings gradually increase. The system continuously reads the values ​​from all pressure sensors and dynamically adjusts the movement speed of the blades on both sides according to the degree of irregularity in the cargo shape. If the pressure on one side rises too quickly, the speed on that side can be appropriately reduced.

[0069] When the sum of pressure from all pressure sensors reaches the predetermined minimum gripping force, which is typically calculated based on the cargo weight, coefficient of friction, and desired safety factor (e.g., a cargo weight of 1.2 kg, a coefficient of friction of 0.45, and a minimum gripping force of 12.0 N), the control system stops the closing action and activates the holding mode. A second fine-grained adjustment component maintains the current gripper opening width, continuously providing a stable gripping force using the holding torque of a motor or cylinder. Subsequently, the embodied intelligent robot lifts the cargo according to a predetermined program and measures the actual weight of the cargo using a six-dimensional force sensor or joint torque estimator on the end effector to confirm successful gripping. If the pressure sensor reading drops abnormally during lifting, indicating cargo slippage, an emergency re-grip or alarm is triggered. The adaptive fitting capability of the flexible gripper relies not only on the feedback from the pressure sensors but also on the elastic deformation of the blade material. When there are localized protrusions on the cargo surface, the flexible material in the protruding areas is compressed more, while the material in the recessed areas is compressed less, resulting in a more uniform pressure distribution between the gripper and the cargo. This avoids high-pressure damage caused by point contact, increases friction, and improves gripping stability.

[0070] The first speed significantly shortens the idle travel time, reducing the single gripping cycle; the second speed ensures gentle contact, preventing high-speed impact damage to irregularly shaped protruding parts or fragile surfaces of goods. The combination of these two speeds balances the cycle time requirements of industrial production lines with the need for cargo protection. By real-time reading of contact pressure and dynamic switching of speed and stopping timing, the flexible gripper can automatically adapt to minute unevenness, tilt, and positional deviations on the cargo surface. Even with a positional error of ±1 mm or an angular deviation of ±2° in the optimal gripping posture, the embodied intelligent robot can still reliably grip, greatly reducing the requirements for hand-eye calibration accuracy.

[0071] In summary, the adaptive grasping method for embodied intelligent robots for irregular goods provided in this application has the following technical effects: By analyzing the target image of the target grasping scene acquired by a depth camera, a three-dimensional voxel of the target is obtained, wherein the target grasping scene refers to a grasping operation scene containing the target irregular goods; the target three-dimensional voxel is input into an implicit grasping inference model to obtain the output result of the implicit grasping inference model, wherein the output result includes at least one candidate grasping pose; the candidate grasping poses are evaluated and compared to obtain the optimal grasping pose, and a grasping simulation is performed on the optimal grasping pose to obtain a simulation result; if the simulation result satisfies predetermined execution constraints, the end effector of the intelligent robot is activated to perform grasping execution on the target irregular goods based on the optimal grasping pose. In other words, multiple candidate grasping poses are quickly generated from three-dimensional voxels through an implicit grasping inference model. After evaluating and obtaining the optimal pose, grasping simulation is performed. Only when the simulation results meet the predetermined execution constraints can the end effector be activated. Before physical execution, infeasible grasping poses are effectively filtered out, the risk of collision and grasping failure is reduced, and the success rate and operational safety of grasping irregular goods are improved.

[0072] Example 2: Based on the same inventive concept as the adaptive grasping method for irregularly shaped goods using a embodied intelligent robot in Example 1, this application also provides an adaptive grasping system for irregularly shaped goods using a embodied intelligent robot. Please refer to the appendix. Figure 2 The embodied intelligent robot adaptive grasping system for irregular goods includes: a target grasping scene analysis module 11, used to analyze target images of the target grasping scene captured by a depth camera to obtain target three-dimensional voxels, wherein the target grasping scene refers to a grasping operation scene containing target irregular goods; a target three-dimensional voxel construction module 12, used to input the target three-dimensional voxels into an implicit grasping inference model to obtain the output result of the implicit grasping inference model, wherein the output result includes at least one candidate grasping pose; a grasping pose evaluation and simulation module 13, used to evaluate and compare the candidate grasping poses to obtain the optimal grasping pose, and perform grasping simulation on the optimal grasping pose to obtain simulation results; and a grasping execution module 14, used to activate the end effector of the intelligent robot to grasp the target irregular goods based on the optimal grasping pose if the simulation result meets predetermined execution constraints.

[0073] Furthermore, the target grasping scene analysis module 11 in the embodied intelligent robot adaptive grasping system for irregular goods is also used to: sequentially extract the first image and the second image from the target image; obtain any voxel in the target grasping scene, and obtain the first distance value of the arbitrary voxel in the first image and the second distance value in the second image; take the average of the first distance value and the second distance value as the arbitrary fusion distance value corresponding to the arbitrary voxel; establish the target three-dimensional voxel based on the correspondence between the arbitrary voxel and the arbitrary fusion distance value; wherein, the first image is a depth image obtained by sampling N points with a hemispherical trajectory at a predetermined height position in the target grasping scene as the first point of the depth camera, and the second image is a depth image obtained by sampling N points with a hemispherical trajectory at a predetermined height position in the target grasping scene as the second point of the depth camera, where N is an integer greater than or equal to 6; wherein, the arbitrary voxel corresponds to an arbitrary cubic region in the target grasping scene, and the side length of the arbitrary cubic region is 4.5 mm.

[0074] Furthermore, the target grasping scene analysis module 11 in the embodied intelligent robot adaptive grasping system for irregular goods is also used to: obtain arbitrary distance values ​​from the arbitrary voxel to the optical center of the depth camera, using the first image as a constraint; obtain arbitrary incident angles between the arbitrary voxel and the acquisition line-of-sight direction of the depth camera, using the first image as a constraint; adjust the arbitrary incident angles by calling a preset weight factor and normalizing them to obtain arbitrary weights; and calculate the first distance value based on the arbitrary weights and the arbitrary distance values.

[0075] Furthermore, the target 3D voxel construction module 12 in the embodied intelligent robot adaptive grasping system for irregular goods is also used for: the implicit grasping inference model includes a shared feature encoder, a first decoder, and a second decoder, and the first decoder and the second decoder are parallel components; inputting the target 3D voxel into the implicit grasping inference model to obtain the output result of the implicit grasping inference model includes: obtaining any grasping point in the target 3D voxel; decoding the grasping pose of the arbitrary grasping point based on the shared feature encoder using the first decoder to obtain an arbitrary grasping quality index; simultaneously, decoding the 3D shape of the arbitrary grasping point based on the shared feature encoder using the second decoder to obtain an arbitrary space occupancy probability; forming arbitrary grasping prediction information for the arbitrary grasping point based on the arbitrary grasping quality index and the arbitrary space occupancy probability; and adding the arbitrary grasping pose corresponding to the arbitrary grasping point to the output result when the arbitrary grasping prediction information meets the predetermined grasping conditions.

[0076] Furthermore, the target 3D voxel construction module 12 in the embodied intelligent robot adaptive grasping system for irregular goods is also used for: the first decoder performing grasping pose decoding on the arbitrary grasping point based on the shared feature encoder to obtain an arbitrary grasping pose; introducing a grasping quality evaluation index to perform feature analysis on the arbitrary grasping pose to obtain grasping quality parameters; and performing label-based normalized weighted processing on the grasping quality parameters to obtain the arbitrary grasping quality index; wherein, the grasping quality evaluation index includes a grasping quality score, a grasping opening width, and a grasping direction.

[0077] Furthermore, the target three-dimensional voxel construction module 12 in the embodied intelligent robot adaptive grasping system for irregular goods is also used for: the predetermined grasping conditions include hard conditions and flexible conditions, wherein the hard conditions refer to the predetermined opening width of the end effector, and the flexible conditions include a predetermined grasping quality index threshold and a predetermined space occupancy probability threshold.

[0078] Furthermore, the grasping pose evaluation and simulation module 13 in the embodied intelligent robot adaptive grasping system for irregular goods is also used for: constructing a target digital twin; obtaining the real-time pose of the end effector and determining the grasping path in combination with the optimal grasping pose; and simulating the grasping path through the target digital twin to obtain the simulation result.

[0079] Furthermore, the grasping pose evaluation and simulation module 13 in the embodied intelligent robot adaptive grasping system for irregular goods is also used to: if the simulation result does not meet the predetermined execution constraint, adjust the grasping path to obtain an adjusted path, and simulate the adjusted path; wherein, the predetermined execution constraint refers to the simulated grasping process without collision and successfully grasping the target irregular goods.

[0080] Furthermore, the grasping execution module 14 in the embodied intelligent robot adaptive grasping system for irregular goods is also used for: the end effector includes a first fine-grained adjustment component, a second fine-grained adjustment component, and a flexible gripper, wherein the first fine-grained adjustment component is used to control the flexible gripper to reach the target irregular goods at a first speed, the second fine-grained adjustment component is used to control the flexible gripper to reach the target irregular goods at a second speed, and the flexible gripper is equipped with a pressure sensor for adaptively conforming to the target irregular goods.

[0081] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The adaptive grasping method and specific examples of the embodied intelligent robot for irregular goods in the foregoing Embodiment 1 are also applicable to the adaptive grasping system of the embodied intelligent robot for irregular goods in this embodiment. Through the foregoing detailed description of the adaptive grasping method of the embodied intelligent robot for irregular goods, those skilled in the art can clearly understand the adaptive grasping system of the embodied intelligent robot for irregular goods in this embodiment. Therefore, for the sake of brevity, it will not be described in detail here.

[0082] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0083] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of this application and its equivalents, this application also intends to include such modifications and variations.

Claims

1. An adaptive grasping method for embodied intelligent robots handling irregularly shaped goods, characterized in that, include: Analyze the target image of the target grasping scene captured by the depth camera to obtain the target three-dimensional voxel, wherein the target grasping scene refers to the grasping operation scene containing the target irregular goods; The target 3D voxel is input into the implicit grasping inference model to obtain the output result of the implicit grasping inference model, wherein the output result includes at least one candidate grasping pose; The candidate grasping poses are evaluated and compared to obtain the optimal grasping pose, and the optimal grasping pose is used to perform a grasping simulation to obtain the simulation results. If the simulation results meet the predetermined execution constraints, the end effector of the intelligent robot is activated to grasp the target irregular goods based on the optimal grasping pose. The process includes evaluating and comparing the candidate grasping poses to obtain the optimal grasping pose, and then performing a grasping simulation on the optimal grasping pose to obtain simulation results, including: Construct the target digital twin; The real-time pose of the end effector is obtained, and the grasping path is determined by combining the optimal grasping pose. The simulation results are obtained by simulating the crawling path using the target digital twin; The process includes evaluating and comparing the candidate grasping poses to obtain the optimal grasping pose, performing a grasping simulation on the optimal grasping pose to obtain simulation results, and then further including: If the simulation results do not meet the predetermined execution constraints, the crawling path is adjusted to obtain an adjusted path, and the adjusted path is simulated. The predetermined execution constraint refers to the simulated grasping process being collision-free and successfully grasping the target irregular goods.

2. The adaptive grasping method for embodied intelligent robots for irregular goods as described in claim 1, characterized in that, Analyze the target image of the target grasping scene captured by the depth camera to obtain the target's three-dimensional voxels, including: The first image and the second image are extracted sequentially from the target image; Obtain any voxel in the target capture scene, and obtain the first distance value of the arbitrary voxel in the first image and the second distance value in the second image; The average of the first distance value and the second distance value is taken as the arbitrary fusion distance value corresponding to the arbitrary voxel; The target three-dimensional voxel is established based on the correspondence between the arbitrary voxel and the arbitrary fusion distance value; Wherein, the first image is a depth image obtained by sampling N points with a hemispherical trajectory at a predetermined height position in the target grasping scene as the first point of the depth camera, and the second image is a depth image obtained by sampling N points with a hemispherical trajectory at a predetermined height position in the target grasping scene as the second point of the depth camera, where N is an integer greater than or equal to 6. Wherein, the arbitrary voxel corresponds to any cubic region in the target grasping scene, and the side length of the arbitrary cubic region is 4.5 mm.

3. The adaptive grasping method for embodied intelligent robots for irregular goods as described in claim 2, characterized in that, Obtaining any voxel in the target capture scene, and obtaining a first distance value of the arbitrary voxel in the first image and a second distance value in the second image, including: Using the first image as a constraint, obtain any distance value from any voxel to the optical center of the depth camera; Using the first image as a constraint, obtain the arbitrary angle of incidence between the arbitrary voxel and the acquisition line of sight of the depth camera; The arbitrary incident angle is adjusted by retrieving a preset weighting factor and then normalized to obtain an arbitrary weight. The first distance value is calculated based on the arbitrary weight and the arbitrary distance value.

4. The adaptive grasping method for embodied intelligent robots for irregular goods as described in claim 1, characterized in that, The implicit crawling inference model includes a shared feature encoder, a first decoder, and a second decoder, wherein the first decoder and the second decoder are parallel components. The target 3D voxel is input into the implicit crawling inference model to obtain its output, which includes: Obtain any grab point in the target three-dimensional voxel; The first decoder performs grasping pose decoding on the arbitrary grasping point based on the shared feature encoder to obtain an arbitrary grasping quality index; Simultaneously, the second decoder performs three-dimensional shape decoding on the arbitrary grasping point based on the shared feature encoder to obtain the probability of arbitrary space occupancy; Based on the arbitrary crawling quality index and the arbitrary space occupancy probability, arbitrary crawling prediction information for the arbitrary crawling point is formed; When the arbitrary grasping prediction information meets the predetermined grasping conditions, the arbitrary grasping pose corresponding to the arbitrary grasping point is added to the output result.

5. The adaptive grasping method for embodied intelligent robots for irregular goods as described in claim 4, characterized in that, The first decoder performs grasping pose decoding on the arbitrary grasping point based on the shared feature encoder to obtain the arbitrary grasping pose; A grasping quality evaluation index is introduced to perform feature analysis on the arbitrary grasping pose to obtain grasping quality parameters; The crawling quality parameters are labeled, normalized, and weighted to obtain the arbitrary crawling quality index; The grasping quality evaluation indicators include grasping quality score, grasping opening width, and grasping direction.

6. The adaptive grasping method for embodied intelligent robots for irregular goods as described in claim 4, characterized in that, The predetermined grasping conditions include hard conditions and flexible conditions. The hard conditions refer to the predetermined opening width of the end effector, and the flexible conditions include a predetermined grasping quality index threshold and a predetermined space occupancy probability threshold.

7. The adaptive grasping method for embodied intelligent robots for irregular goods as described in claim 1, characterized in that, The end effector includes a first fine-grained adjustment component, a second fine-grained adjustment component, and a flexible gripper. The first fine-grained adjustment component controls the flexible gripper to reach the target irregular cargo at a first speed, and the second fine-grained adjustment component controls the flexible gripper to reach the target irregular cargo at a second speed. The flexible gripper is equipped with a pressure sensor for adaptively conforming to the target irregular cargo.

8. An embodied intelligent robot adaptive grasping system for irregularly shaped goods, characterized in that, The steps for implementing the adaptive grasping method for irregular cargo using an embodied intelligent robot according to any one of claims 1 to 7, wherein the adaptive grasping system for irregular cargo using an embodied intelligent robot comprises: The target grasping scene analysis module is used to analyze the target image of the target grasping scene captured by the depth camera to obtain the target three-dimensional voxel. The target grasping scene refers to the grasping operation scene containing the target irregular goods. A target 3D voxel construction module is used to input the target 3D voxel into an implicit grasping inference model to obtain the output result of the implicit grasping inference model, wherein the output result includes at least one candidate grasping pose; The grasping pose evaluation and simulation module is used to evaluate and compare the candidate grasping poses to obtain the optimal grasping pose, and to perform grasping simulation on the optimal grasping pose to obtain simulation results. The grasping execution module is used to activate the end effector of the intelligent robot to grasp the target irregular goods based on the optimal grasping pose if the simulation result meets the predetermined execution constraints.

Images