Grabbing control method and system of 3D sorting machine

Abstract

The invention discloses a grabbing control method and system of a 3D sorting machine, and relates to the related field of material sorting, and the grabbing control method comprises the steps that multi-angle image data of a target material is acquired through a binocular structured light camera, three-dimensional reconstruction is carried out, and a material three-dimensional model is obtained; performing matching analysis based on the material three-dimensional model and the material order data to obtain material sorting grid positions and material property information; based on the working characteristic data of the 3D sorting machine, grabbing analysis is conducted on the material three-dimensional model and the material property information, and a target material grabbing point and target material grabbing force are determined; constructing a material grabbing path space; and a grabbing path is planned, a target material grabbing path is generated, and the 3D sorting machine executes material grabbing and compensation closed-loop control. The technical problem that grabbing control of an existing sorting machine is low in control precision and reliability is solved, and the technical effect of improving the grabbing control precision and reliability of the sorting machine is achieved.

Description

Technical Field

[0001] The present application relates to the field of material sorting, and in particular to a grasping control method and system for a 3D sorting machine. Background Art

[0002] In the field of automated logistics and warehousing, efficient and accurate material sorting is a key link to ensure the smooth operation of the entire system and improve production efficiency. It is of great significance to reducing costs and improving corporate competitiveness. At present, the solution to the problem of material sorting mainly relies on traditional 2D visual recognition combined with simple mechanical grasping methods. The material image information is obtained through a 2D camera, and the grasping position and force are determined according to preset rules. However, since traditional methods can only obtain two-dimensional information of the material, they cannot accurately perceive the three-dimensional shape, material characteristics and other key factors of the material. As a result, when faced with materials with irregular shapes and large material differences, problems such as grasping failure and material damage caused by improper grasping force are prone to occur.

[0003] In the current related technologies, the grasping control of the sorting machine has technical problems of low control accuracy and reliability. Summary of the Invention

[0004] The present application provides a grasping control method and system for a 3D sorting machine, which uses a binocular structured light camera to collect multi-angle image data of the target material, obtains a three-dimensional model of the material through three-dimensional reconstruction, then reads the material order data, combines the three-dimensional model for matching analysis, obtains the material sorting grid position and material attribute information, and then grasps and analyzes the three-dimensional model and material attribute information based on the working characteristic data of the 3D sorting machine to determine the grasping point and grasping force of the target material. Then, the material sorting grid position, the target material grasping point and the three-dimensional model are used as constraint parameters to construct a material grasping path space, and finally plans the grasping path within the space to generate the target material grasping path. The 3D sorting machine executes material grasping and implements technical means such as compensation closed-loop control based on this, thereby solving the technical problems of low control accuracy and reliability of the grasping control of the existing sorting machine, and achieving the technical effect of improving the accuracy and reliability of the grasping control of the sorting machine.

[0005] The present application provides a grasping control method for a 3D sorting machine, comprising: acquiring multi-angle image data of a target material through a binocular structured light camera, performing three-dimensional reconstruction based on the multi-angle image data, and obtaining a three-dimensional model of the material; reading material order data, performing matching analysis based on the material three-dimensional model and the material order data, and obtaining material sorting grid position and material material attribute information; grasping and analyzing the material three-dimensional model and the material material attribute information based on working characteristic data of the 3D sorting machine, and determining a target material grasping point and a target material grasping force; using the material sorting grid position, the target material grasping point, and the material three-dimensional model as constraint parameters, constructing a material grasping path space of the 3D sorting machine; performing grasping path planning in the material grasping path space, and generating a target material grasping path, wherein the 3D sorting machine performs material grasping and compensation closed-loop control based on the target material grasping point, the target material grasping force, and the target material grasping path.

[0006] In a possible implementation, the three-dimensional model of the material is obtained by performing the following processing: initializing a Gaussian filter to perform Gaussian filtering and contrast enhancement on the multi-angle image data to obtain usable multi-angle image data; performing structured light decoding and image registration on the usable multi-angle image data to obtain a multi-view registration depth map; loading the calibration parameters of the binocular structured light camera and the multi-view registration depth map to perform point cloud conversion and fusion to obtain global material point cloud data; performing gridding processing and three-dimensional surface reconstruction based on the global material point cloud data to obtain a three-dimensional model of the material.

[0007] In a possible implementation, the multi-view registration depth map is obtained by performing the following processing: performing phase calculation extraction and phase unwrapping on each image in the available multi-view image data to obtain multi-view absolute phase data; performing depth conversion on the multi-view absolute phase data based on the calibration parameters of the binocular structured light camera to obtain multiple single-view depth maps; performing feature point extraction on each view image in the multiple single-view depth maps to obtain multiple image feature point sets; and using the multiple image feature point sets to align the multiple single-view depth maps to obtain the multi-view registration depth map.

[0008] In a possible implementation, the three-dimensional model of the material is obtained, and the following processing is performed: outlier filtering and normal estimation calculation are performed on the global material point cloud data to generate global point cloud normal information; Poisson reconstruction parameters are set, and the global material point cloud data is structurally divided based on the Poisson reconstruction parameters to construct a multi-level point cloud octree set; according to the global point cloud normal information, the vector field is calculated in the nodes of the multi-level point cloud octree set and the Poisson equation is solved to obtain an implicit function approximate solution; based on the implicit function approximate solution, isosurface extraction and mesh post-processing are performed to obtain the three-dimensional model of the material after three-dimensional surface reconstruction.

[0009] In a possible implementation, the target material grasping point and the target material grasping force are determined by performing the following processing: performing surface feature recognition on the three-dimensional model of the material to obtain a set of candidate grasping points; matching and optimizing the set of candidate grasping points based on the working characteristic data of the 3D sorting machine to determine the target material grasping point; if the target material grasping point is zero, solving the center of gravity of the three-dimensional model of the material to obtain the material center of gravity information, and using the material center of gravity information as the target material grasping point; performing grasping and parsing on the three-dimensional model of the material and the material material attribute information based on the working characteristic data of the 3D sorting machine to obtain the target material grasping force.

[0010] In a possible implementation, the target material grasping force is obtained by performing the following processing: determining the grasper restriction conditions based on the working characteristic data of the 3D sorting machine; decomposing the grasping force requirements of the material three-dimensional model and the material material attribute information to obtain the grasping gravity component, grasping inertia component and grasping safety component; and determining the maximum values ​​of the grasping gravity component, grasping inertia component and grasping safety component based on the grasper restriction conditions to determine the target material grasping force.

[0011] In a possible implementation, the material grabbing path space of the 3D sorting machine is constructed by performing the following processing: initializing the three-dimensional space of the grabbing path according to the working range of the 3D sorting machine; using the material sorting grid position, the target material grabbing point and the three-dimensional model of the material as constraint parameters, constraining the three-dimensional space of the grabbing path to construct the material grabbing path space.

[0012] In a possible implementation, the target material grabbing path is generated by performing the following processing: constructing a grabbing path cost function according to the material grabbing planning target; using the grabbing path cost function to perform path planning and cost evaluation in the material grabbing path space, and determining the target material grabbing path with the minimum cost.

[0013] In a possible implementation, the 3D sorting machine performs material grabbing and compensation closed-loop control based on the target material grabbing point and target material grabbing force, as well as the target material grabbing path, and performs the following processing: the 3D sorting machine performs material grabbing feedback based on the target material grabbing point and target material grabbing force, as well as the target material grabbing path, and obtains material grabbing deviation parameters; and performs compensation adjustment and closed-loop control on the target material grabbing force and the target material grabbing path based on the material grabbing deviation parameters.

[0014] The present application also provides a grasping control system for a 3D sorting machine, including: a material three-dimensional model construction module, which is used to acquire multi-angle image data of a target material through a binocular structured light camera, and perform three-dimensional reconstruction based on the multi-angle image data to obtain a three-dimensional model of the material; a matching analysis module, which is used to read material order data, perform matching analysis based on the material three-dimensional model and the material order data, and obtain material sorting grid position and material material attribute information; a grasping analysis module, which is used to grasp and analyze the material three-dimensional model and the material material attribute information based on the working characteristic data of the 3D sorting machine, and determine the target material grasping point and the target material grasping force; a material grasping path space construction module, which is used to use the material sorting grid position, the target material grasping point and the material three-dimensional model as constraint parameters to construct the material grasping path space of the 3D sorting machine; a material grasping module, which is used to perform grasping path planning in the material grasping path space to generate a target material grasping path, and the 3D sorting machine performs material grasping and compensation closed-loop control based on the target material grasping point and the target material grasping force, as well as the target material grasping path.

[0015] The present application proposes a grasping control method and system for a 3D sorting machine. First, a binocular structured light camera is used to acquire multi-angle image data of a target material. Based on the multi-angle image data, three-dimensional reconstruction is performed to obtain a three-dimensional model of the material. Then, material order data is read and matching analysis is performed based on the three-dimensional model and the material order data to obtain material sorting grid positions and material material attribute information. Then, based on the operating characteristic data of the 3D sorting machine, the three-dimensional model and the material material attribute information are grasped and analyzed to determine the target material grasping point and target material grasping force. The material sorting grid position, the target material grasping point, and the three-dimensional model are then used as constraint parameters to construct a material grasping path space for the 3D sorting machine. Finally, grasping path planning is performed within the material grasping path space to generate a target material grasping path. The 3D sorting machine performs material grasping and compensation closed-loop control based on the target material grasping point, target material grasping force, and target material grasping path. This achieves the technical effect of improving the accuracy and reliability of the sorting machine's grasping control. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings of the embodiments of the present invention are briefly introduced below. Flowcharts are used in this application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed in precise order. Instead, various steps may be processed in reverse order or simultaneously as needed. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0017] Figure 1 A flowchart of a grasping control method for a 3D sorting machine provided in an embodiment of the present application.

[0018] Figure 2 A schematic structural diagram of a gripping control system for a 3D sorting machine provided in an embodiment of the present application.

[0019] Description of the accompanying drawings: material three-dimensional model construction module 10, matching analysis module 20, grasping analysis module 30, material grasping path space construction module 40, material grasping module 50. DETAILED DESCRIPTION

[0020] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below.

[0021] In order to make the purpose, technical solutions and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0022] In the following description, reference is made to “some embodiments” which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict. The terms “including” and “having” and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. The terms used herein are for the purpose of describing the embodiments of this application only.

[0023] The embodiment of the present application provides a grasping control method for a 3D sorting machine, such as Figure 1 As shown, the method includes: Step S100: acquiring multi-angle image data of a target material through a binocular structured light camera, and performing three-dimensional reconstruction based on the multi-angle image data to obtain a three-dimensional model of the material.

[0024] Specifically, a binocular structured light camera is a camera system that combines binocular vision and structured light technology to acquire three-dimensional information about objects. Using a binocular camera system with structured light technology, the system uses two cameras to capture images of a target material from different angles. Structured light technology projects a specific light pattern (such as stripes or dots) onto the material surface and calculates the object's depth information based on the deformation of the light pattern. The binocular camera captures multi-angle images of the target material at a constant frequency (for example, 30Hz). The image data captured by each camera is synchronously transmitted to a processing unit. Computer vision algorithms (such as stereo matching) are used to process the images captured by the binocular cameras and, combined with the depth information from the structured light, reconstruct a three-dimensional model of the target material. For example, the stereo matching functions cv2.StereoBM_create() or cv2.StereoSGBM_create() in the OpenCV library are used to calculate a depth map. A complete three-dimensional model is then generated using a 3D reconstruction algorithm (such as the ICP algorithm).

[0025] For example, consider the Intel RealSense D435i stereo depth camera, which has a built-in structured light module and can simultaneously provide color and depth images. Assuming the target material is a small part, the stereo camera captures images at a 30Hz frequency, acquiring 30 pairs of image data per second. Using the OpenCV cv2.StereoSGBM_create() function, parameters such as minDisparity and numDisparities are set to calculate the depth map. A 3D reconstruction algorithm is then used to generate a 3D model of the part.

[0026] In one possible implementation, the three-dimensional model of the material is obtained, and step S100 further includes step S110, initializing a Gaussian filter to perform Gaussian filtering and contrast enhancement on the multi-angle image data to obtain usable multi-angle image data. Specifically, the parameters of the Gaussian filter are set, such as the standard deviation (σ) and the filter size (window size). The standard deviation determines the smoothness of the filter, and the window size determines the range of action of the filter. A Gaussian filter is applied to the multi-angle image data to remove noise from the image and smooth the image. Gaussian filtering is a linear filtering method that can effectively reduce high-frequency noise in the image while retaining the main features of the image. The contrast of the image is improved by histogram equalization or other contrast enhancement algorithms (such as linear contrast stretching), making the features of the target object in the image more obvious.

[0027] For example, the Gaussian filter parameters are set to a standard deviation of σ = 1.5 and a filter size of 5 × 5. Use the cv2.GaussianBlur() function in the OpenCV library to perform Gaussian filtering on the image. Use the cv2.equalizeHist() function in OpenCV to perform histogram equalization on the grayscale image, or use the cv2.convertScaleAbs() function to perform linear contrast stretching.

[0028] Step S120: Structured light decoding and image registration are performed on the available multi-angle image data to obtain a multi-view registered depth map. Specifically, the structured light pattern is decoded from the image to extract depth information. Structured light decoding includes a phase unwrapping algorithm (such as a four-step phase shift algorithm) that calculates the depth of the object surface by analyzing the deformation of the light pattern. The available multi-angle image data is aligned to ensure spatial consistency of images from different perspectives. Image registration can use feature point matching methods (such as ORB or SIFT) or optical flow-based methods.

[0029] For example, using a four-step phase shift algorithm, four structured light images with different phase offsets are captured to calculate the phase value of each pixel and decode the depth information. OpenCV's cv2.ORB_create() is used to extract feature points, followed by feature matching using cv2.BFMatcher. Finally, cv2.findHomography() is used to calculate the homography matrix and align the multi-angle images.

[0030] Step S130 loads the calibration parameters of the binocular structured light camera and the multi-view registered depth map for point cloud conversion and fusion to generate global material point cloud data. Specifically, the calibration parameters of the binocular structured light camera are loaded, including camera intrinsic parameters (focal length, principal point coordinates) and extrinsic parameters (rotation matrix and translation vector). These parameters are used to convert image coordinates into three-dimensional space coordinates. Based on the calibration parameters and the multi-view registered depth map, the depth information of each pixel is converted into point cloud data in three-dimensional space. The multi-view point cloud data is fused to generate global material point cloud data. Point cloud fusion can use the iterative closest point (ICP) algorithm or other point cloud registration algorithms.

[0031] Step S140: Meshing and 3D surface reconstruction are performed based on the global material point cloud data to obtain a 3D model of the material. Specifically, the global material point cloud data is meshed to generate a triangular mesh. Meshing can utilize a Marching Cubes algorithm or other surface reconstruction algorithms. Surface reconstruction is performed on the meshed point cloud data to generate a complete 3D model. Surface reconstruction can utilize a Poisson reconstruction algorithm or other advanced reconstruction algorithms.

[0032] For example, the Marching Cubes algorithm in the VTK (Visualization Toolkit) library is used to convert global material point cloud data into a triangular mesh. The Poisson reconstruction algorithm in the PCL library is then used to reconstruct the surface of the meshed point cloud data, generating a smooth 3D model. This implementation utilizes a series of technical approaches, including Gaussian filtering, contrast enhancement, structured light decoding, image registration, point cloud conversion and fusion, meshing processing, and surface reconstruction, to gradually optimize the process from image acquisition to 3D model reconstruction. These technical approaches work together to improve the accuracy and reliability of 3D model reconstruction, providing high-quality 3D data support for subsequent grasping control.

[0033] In one possible implementation, step S120 of obtaining a multi-view registered depth map further includes step S121 of performing phase calculation extraction and phase unwrapping on each image in the available multi-angle image data to obtain multi-angle absolute phase data. Specifically, a phase unwrapping algorithm (such as a four-step phase shift algorithm) in structured light technology is used to extract phase information from each image. This phase information reflects the deformation of the structured light pattern on the object surface and is used to calculate depth information. Because the phase information of the structured light pattern exhibits 2π periodicity, a phase unwrapping algorithm (such as a quality-guided phase unwrapping algorithm) is required to unwrap the phase information into an absolute phase to eliminate phase ambiguity.

[0034] For example, a four-step phase shift algorithm is used to capture four structured light images with different phase offsets and calculate the phase value of each pixel. A quality-guided phase unwrapping algorithm analyzes the quality map of the phase image and gradually unwraps the phase to obtain absolute phase data.

[0035] Step S122, based on the calibration parameters of the binocular structured light camera, the multi-angle absolute phase data is depth-converted to obtain multiple single-view depth maps. Specifically, the calibration parameters of the binocular structured light camera are loaded, including the camera intrinsic parameters (focal length, principal point coordinates) and extrinsic parameters (rotation matrix and translation vector). According to the calibration parameters, the absolute phase data is converted into depth information. The depth conversion uses a triangulation formula to calculate the phase difference and camera parameters to obtain the depth value of each pixel. For example, using the formula Depth= , where B is the baseline distance, f is the focal length, and Δφ is the phase difference.

[0036] Step S123: Feature point extraction is performed on each view image in the multiple monoscopic depth maps to obtain multiple sets of image feature points. Specifically, feature points are extracted from each monoscopic depth map using a computer vision algorithm (such as ORB, SIFT, or SURF). Feature points are salient points in an image and are used for image registration and alignment. For example, for each monoscopic depth map, feature points are extracted using cv2.ORB_create(), and descriptors for the feature points are calculated using cv2.compute().

[0037] In step S124, the multiple single-view depth maps are registered and aligned using the multiple image feature point sets to obtain the multi-view registered depth map. Specifically, a feature point matching algorithm (such as BFMatcher or FLANN) is used to match feature points from different viewpoints to find corresponding relationships. Based on the matched feature points, a homography matrix is ​​calculated and used to align the images. Image registration ensures spatial consistency between the multi-view depth maps. For example, OpenCV's cv2.BFMatcher is used for feature point matching. The homography matrix is ​​calculated using cv2.findHomography(), and the image is transformed using cv2.warpPerspective() to achieve registration and alignment. This implementation method gradually optimizes the process of generating multi-view registered depth maps from available multi-view image data through a series of technical approaches, including phase calculation and extraction, phase unwrapping, depth conversion, feature point extraction, and image registration and alignment. These technical approaches work together to improve the accuracy and consistency of depth maps, providing high-quality data support for subsequent point cloud fusion and 3D reconstruction. This optimization method can effectively reduce errors and improve the robustness and accuracy of the system.

[0038] In one possible implementation, step S140 of obtaining a 3D material model further includes step S141 of performing outlier filtering and normal estimation on the global material point cloud data to generate global point cloud normal information. Specifically, outliers are removed from the global material point cloud data using statistical methods or neighborhood-based filtering algorithms (such as voxel grid filtering or radius-based filtering). Outliers are typically caused by noise or measurement errors, affecting subsequent processing and reconstruction accuracy. The normal direction of each point is estimated using local neighborhood information (such as k-nearest neighbors). This normal information provides local surface orientation information.

[0039] For example, use the pcl::StatisticalOutlierRemoval class in the PCL library to perform outlier filtering, set threshold parameters (such as standard deviation multiples), and use the pcl::NormalEstimation class in the PCL library to calculate the normal direction of each point through k-nearest neighbor search.

[0040] Step S142 sets Poisson reconstruction parameters and, based on these parameters, structures the global material point cloud data to construct a multi-level point cloud octree set. Specifically, key Poisson reconstruction parameters are set, such as depth (indicating the number of octree layers) and scale (controlling surface smoothness). The global material point cloud data is divided into a multi-level octree structure. An octree is a spatial partitioning method that efficiently organizes and processes point cloud data, facilitating the subsequent solution of the Poisson equation.

[0041] For example, set the depth to 8 (indicating an octree with 8 levels) and the scale to 1.0 (indicating a moderately smooth surface). Use the pcl::octree::OctreePointCloud class in the PCL library to construct a multi-level point cloud octree collection.

[0042] In step S143, a vector field is calculated at the nodes of the multi-level point cloud octree based on the global point cloud normal information and the Poisson equation is solved to obtain an approximate solution to the implicit function. Specifically, at each node in the octree, a vector field is calculated based on the point cloud normal information. The vector field represents the gradient information of the local surface. The Poisson equation is solved to obtain an approximate solution to the implicit function. The Poisson equation is solved using numerical methods (such as the finite difference method), and the zero isosurface of the implicit function is the reconstructed surface.

[0043] For example, using the pcl::Poisson class in the PCL library, in each node of the octree, the vector field is calculated based on the normal information of the point cloud, and the Poisson equation is solved by the finite difference method to obtain an approximate solution to the implicit function.

[0044] Step S144: Isosurface extraction and mesh post-processing are performed based on the implicit function approximate solution to obtain a 3D model of the material after 3D surface reconstruction. Specifically, a Marching Cubes algorithm or other isosurface extraction algorithm is used to extract the zero isosurface, i.e., the reconstructed surface, from the implicit function. The extracted mesh is then optimized, such as by smoothing, simplifying, and removing small holes, to improve mesh quality and visual quality.

[0045] For example, the vtkMarchingCubes class in the VTK library is used to extract the zero isosurface of the implicit function. The extracted mesh is then smoothed using mesh smoothing algorithms in the PCL library (such as pcl::MeshSmoothingLaplacian). This implementation utilizes a series of technical approaches, including outlier filtering, normal estimation, Poisson reconstruction parameter setting, octree structure partitioning, vector field calculation, Poisson equation solution, isosurface extraction, and mesh post-processing, to gradually optimize the reconstruction process from global material point cloud data to the material 3D model. These technical approaches work together to improve the accuracy and quality of the 3D model reconstruction.

[0046] Step S200 , reading the material order data, performing matching analysis based on the material three-dimensional model and the material order data, and obtaining the material sorting grid position and material material attribute information.

[0047] Specifically, material order data is read through an industrial-grade database management system (such as MySQL or MongoDB). Material order data contains information such as the material type, quantity, target sorting grid position, and material properties. First, the order data related to the current material to be sorted is read from the database. This data provides basic information about the material, including its material properties. Computer vision algorithms (such as feature matching algorithms) and machine learning algorithms (such as classifiers) are used to analyze the material's three-dimensional model to identify the material's type and material properties. For example, the geometric features (such as shape and size) and surface features (such as texture and color) of the material's three-dimensional model are extracted and matched with the preset material template. Based on the matching results, the corresponding sorting grid position and material property information are obtained from the material order data.

[0048] For example, a MySQL database is used to store material order data, and order information is retrieved through SQL queries. OpenCV's cv2.ORB_create() function is used to extract feature points from the material's 3D model, and then cv2.BFMatcher is used for feature matching. If the match exceeds a preset threshold, the material is considered the target, and its sorting grid location and material properties are determined based on the information in the material order data.

[0049] Step S300 : performing grasping and analysis on the three-dimensional model of the material and the material property information of the material based on the working characteristic data of the 3D sorting machine to determine the target material grasping point and the target material grasping force.

[0050] Specifically, the operating characteristic data of a 3D sorter includes the range of motion of the robotic arm, the type of gripping tool (e.g., suction cup, gripper), and gripping capacity (e.g., maximum gripping force). Mechanical and kinematic models are used to analyze the material's 3D model and material properties. Optimization algorithms (e.g., genetic algorithms or particle swarm optimization) are then used to search for the optimal solution within the feasible range of gripping points and gripping forces, determining the optimal gripping point and gripping force. For example, the optimal gripping point of the robotic arm is calculated based on the material's shape and center of gravity, and the appropriate gripping force is determined based on material properties (e.g., hardness and elastic modulus).

[0051] For example, a 3D sorting machine uses a six-axis robotic arm with a vacuum cup at its end and a maximum gripping force of 50N. Finite element analysis software (such as ANSYS) is used to simulate the material's mechanical properties and calculate the appropriate gripping force based on material properties (e.g., a hardness of 200HV for metal and 50HV for plastic). A genetic algorithm is then used to search for the optimal gripping point among multiple candidate gripping points on the material surface, ensuring that the material does not slip or become damaged during the gripping process.

[0052] In one possible implementation, determining the target material grasping point and target material grasping force in step S300 further includes step S310, which performs surface feature recognition on the 3D material model to obtain a set of candidate grasping points. Specifically, computer vision algorithms and geometric analysis methods are used to identify surface features of the 3D material model, such as edges, grooves, and planes. These feature points represent ideal locations for grasping operations. Based on the identified surface features, a set of candidate grasping points is generated. These points meet certain geometric requirements, such as being located on a flat surface or within a groove and being easily accessible to the robotic arm.

[0053] For example, use the pcl::RegionGrowing algorithm in the PCL library to identify planar regions on the material surface, or use OpenCV's edge detection algorithm (such as Canny edge detection) to identify edge features. Uniformly sample the identified planar regions or edge features to generate a set of candidate grasping points.

[0054] Step S320 involves matching and optimizing the candidate gripping point set based on the 3D sorter's operating characteristic data to determine the target material gripping point. Specifically, the 3D sorter's operating characteristic data is loaded, including the robot's range of motion, gripping tool type (e.g., suction cup, gripper), and gripping capacity (e.g., maximum gripping force). Based on this operating characteristic data, the candidate gripping point set is screened and optimized. For example, points outside the robot's range of motion are excluded, or points that are most suitable for the gripping tool are selected.

[0055] For example, suppose a 3D sorting machine uses a six-axis robotic arm with a vacuum suction cup at the end, a maximum gripping force of 50N, and a range of motion within a spherical space with a radius of 1 meter. An optimization algorithm (such as a genetic algorithm or particle swarm optimization) is used to search for the optimal gripping point among a set of candidate gripping points. The optimization goal might be to minimize the distance from the gripping point to the initial position of the robotic arm while ensuring that the gripping point is within the range of motion of the robotic arm.

[0056] In step S330, if the target material grabbing point is zero, the center of gravity of the 3D material model is calculated to obtain the material's center of gravity information, which is then used as the target material grabbing point. Specifically, if the candidate grabbing point set is empty (i.e., there are no suitable grabbing points), the center of gravity of the 3D material model is calculated. The center of gravity can be calculated using geometric calculations or numerical integration methods. The center of gravity information is used as the target material grabbing point because the center of gravity is typically the material's equilibrium point, making it suitable for grabbing operations.

[0057] For example, use the pcl::compute3DCentroid function in the PCL library to calculate the center of gravity of the material's 3D model. Use the center of gravity coordinates as the target material grasping point to ensure the robot arm can grasp the material stably.

[0058] In step S340, the 3D material model and material property information are analyzed based on the 3D sorter's operating characteristic data to determine the target material gripping force. Specifically, the target material gripping force is calculated using a mechanical model (such as a friction model or elasticity model) based on the material's 3D model and material property information, combined with the 3D sorter's operating characteristic data. The gripping force needs to be adjusted based on the material's weight, shape, and material properties. For example, a higher gripping force is required for heavier metal materials, while a lower gripping force can be used for lighter plastic materials.

[0059] For example, suppose the material is a metal part weighing 2 kg and having a material hardness of 200 HV. Based on the mechanical model, a gripping force of 30 N is calculated to ensure that the material does not slip or become damaged during the gripping process. If the material is a plastic part weighing 0.5 kg and having a material hardness of 50 HV, the gripping force can be adjusted to 10 N to avoid material damage. This implementation method uses a series of technical methods, including surface feature recognition, matching optimization, center of gravity solution, and grip analysis, to gradually optimize the process of determining the target material's gripping point and gripping force. These technical methods work together to improve the accuracy and reliability of the gripping operation.

[0060] In one possible implementation, step S340 of obtaining the target material gripping force further includes step S341 of determining gripper constraints based on the operating characteristic data of the 3D sorter. Specifically, the operating characteristic data of the 3D sorter is loaded, including the gripper's maximum gripping force, minimum gripping force, gripper type (e.g., suction cup, clamp), gripper range of motion, etc. Based on the operating characteristic data, gripper constraints are determined. These constraints include the maximum gripping force, minimum gripping force, gripper range of motion, gripper friction coefficient, etc.

[0061] For example, suppose the gripper of a 3D sorting machine is a vacuum suction cup with a maximum gripping force of 50N and a minimum gripping force of 10N. The gripper's range of motion is a spherical space with a radius of 1 meter, and the coefficient of friction is 0.5. Based on this data, the constraints of the gripper are determined to be: maximum gripping force: 50N, minimum gripping force: 10N, and friction coefficient: 0.5.

[0062] Step S342 decomposes the grasping force requirement based on the three-dimensional model of the material and the material property information to obtain the grasping gravity component, the grasping inertia component, and the grasping safety component. Specifically, the grasping force required during the grasping process is calculated based on the three-dimensional model and material property information of the material. The grasping force requirement can be decomposed into three main components: the grasping gravity component is used to overcome the gravity of the material to ensure that the material does not fall; the grasping inertia component is used to overcome the inertia of the material during movement to ensure the stability of the grasping process; and the grasping safety component is used to ensure the safety of the grasping process by adding a safety factor to prevent the material from sliding or falling due to insufficient grasping force.

[0063] For example, assuming the weight of the material is 2 kg and the acceleration due to gravity is 9.8 m / s 2 , the grabbing gravity component is: F 重力 =m·g=2×9.8=19.6N, assuming that the maximum acceleration of the material during movement is 2m / s 2 , the grasping inertia component is: F 惯性=m·a=2×2=4N, assuming the safety factor is 1.2, the grasping safety component is: F 安全 =F 重力 1.2=19.6×1.2=23.52N.

[0064] In step S343, the maximum values ​​of the grasping gravity, inertia, and safety components are determined based on the gripper constraints to determine the target material grasping force. Specifically, the maximum values ​​of the grasping gravity, inertia, and safety components are determined based on the gripper constraints. The target material grasping force should meet the gripper constraints and ensure the stability and safety of the gripping process. The maximum value of these three components is selected as the target material grasping force, ensuring that this value is within the gripper constraints. For example, the maximum value of the grasping gravity, inertia, and safety components is 23.52 N, which is within the gripper constraints (10 N to 50 N). Therefore, the target material grasping force is 23.52 N. This implementation method gradually optimizes the process of determining the target material grasping force through a series of technical measures, including determining gripper constraints, decomposing the grasping force requirements, and determining the maximum value. These technical measures work together to improve the accuracy and reliability of the target material grasping force calculation.

[0065] Step S400 : Using the material sorting grid position, the target material grabbing point and the material three-dimensional model as constraint parameters, a material grabbing path space of the 3D sorting machine is constructed.

[0066] Specifically, the material sorting grid location is the final destination for the material and serves as one of the endpoints of path planning. The target material grabbing point is the location the robot needs to reach and is a key intermediate point in path planning. The 3D material model provides information about the material's shape and dimensions, which is used for collision detection and path planning. The material sorting grid location, the target material grabbing point, and the 3D material model serve as constraints for path planning. A kinematic model and collision detection algorithm are used to construct the material grabbing path space. The material grabbing path space represents the range of motion within the robot arm in 3D space, constrained by the object's position, the grabbing point, and collision avoidance constraints. For example, inverse kinematics is used to calculate the robot arm's trajectory from its initial position to the target material grabbing point. A collision detection algorithm (such as V-REP simulation software) is used to ensure that the robot arm avoids collisions with the surrounding environment during movement. After grabbing the material, the robot arm needs to move the material to the sorting grid location. This stage of path planning also requires collision detection. A path planning algorithm (such as the A* algorithm or the RRT algorithm) is used to search for the optimal path within the material grabbing path space. These algorithms are able to take into account the kinematic constraints of the robotic arm and obstacles in the environment to ensure the feasibility and safety of the path.

[0067] In one possible implementation, step S400 of constructing the material grasping path space for the 3D sorter further includes step S410 of initializing the three-dimensional grasping path space based on the working range of the 3D sorter. Specifically, the working range of the 3D sorter is obtained, which is determined by the kinematic model of the robotic arm. The working range is a three-dimensional space within which all movements of the robotic arm must occur. Based on the working range of the robotic arm, a three-dimensional space is initialized. This space serves as the basis for subsequent path planning. This space can be represented by a three-dimensional grid or octree for subsequent path planning and collision detection.

[0068] For example, suppose a 3D sorter is a six-axis robotic arm whose working range is a spherical space with a radius of 1 meter. This spherical space is initialized using an octree data structure. Each node in the octree represents a spatial region, and the entire spherical space is initially divided into multiple small cubic regions.

[0069] In step S420, the material sorting grid position, the target material grabbing point, and the material's 3D model are used as constraint parameters to perform constraint correction on the 3D grasping path space and construct the material grabbing path space. Specifically, the material sorting grid position is used as the target endpoint for path planning, ensuring that the path planning endpoint is the material sorting grid position. The target material grabbing point is used as a key intermediate point in path planning to ensure that the robot arm can reach the grabbing point and grab the material. The 3D model of the material is used as the basis for collision detection to ensure that the robot arm does not collide with the material or the surrounding environment during the grasping and moving process. Based on these constraint parameters, the initialized 3D grasping path space is corrected. The correction process includes: limiting the path planning area to the robot arm's working range and excluding all inaccessible areas; marking all areas in the 3D grasping path space that may collide with the material or the environment and excluding these areas from the path planning; and using a path planning algorithm (such as the A* algorithm or the RRT algorithm) to search for the optimal path in the corrected 3D space.

[0070] For example, assume that the working range of the robot arm is a spherical space with a radius of 1 meter, and exclude all areas outside this range. Use the collision detection algorithm in the PCL library to mark all areas that may collide with materials or the environment. For example, if the material is a cuboid, calculate the collision areas between the robot arm and the cuboid during the grasping and moving process, and mark these areas as unreachable. Use the RRT algorithm to search for the optimal path from the initial position of the robot arm to the target material grasping point, and then to the material sorting grid position in the corrected three-dimensional space. This implementation method gradually optimizes the construction process of the material grasping path space of the 3D sorting machine by initializing the three-dimensional space of the grasping path and constraining the three-dimensional space of the grasping path. These technical means work together to improve the accuracy and reliability of path planning.

[0071] Step S500: Performing grabbing path planning in the material grabbing path space to generate a target material grabbing path. The 3D sorting machine performs material grabbing and compensation closed-loop control based on the target material grabbing point, target material grabbing force, and the target material grabbing path.

[0072] Specifically, a path planning algorithm (such as the RRT algorithm) is used to generate a complete path from the initial position of the robotic arm to the target material grasping point and then to the material sorting grid position. During the grasping process, sensors (such as force sensors and visual sensors) are used to monitor the grasping status in real time, and a PID controller or fuzzy controller is used for compensatory closed-loop control. For example, if a material position deviation is detected during the grasping process, it is corrected by adjusting the motion parameters of the robotic arm. The 3D sorter performs the grasping action according to the planned path and grasping parameters, and accurately places the material in the sorting grid.

[0073] In one possible implementation, the target material grasping path is generated, and step S500 further includes step S510, which constructs a grasping path cost function based on the material grasping planning target. Specifically, the grasping path cost function is a mathematical model for evaluating the quality of the path, which comprehensively considers multiple factors, such as path length, collision risk, grasping stability, etc. Among them, the shorter the path, the shorter the movement time of the robot arm and the higher the efficiency. The lower the possibility of collision in the path, the safer the path. The higher the stability of the grasping point in the path, the higher the grasping success rate. The smaller the change in acceleration and velocity in the path, the better the dynamic performance of the robot arm.

[0074] For example, path length can be calculated as the sum of the Euclidean distances between all points on the path. Collision risk can be assessed by the minimum distance between each point on the path and an obstacle. The smaller the distance, the higher the collision risk. Grasping stability can be assessed by the angle between the normal direction of the grasping point and the grasping direction of the robot arm. The smaller the angle, the higher the stability. Dynamic performance can be assessed by the rate of change of the velocity and acceleration at each point on the path. The smaller the rate of change, the better the dynamic performance. The grasping path cost function C(p) can be: C(p) = w1·L(p) + w2·R(p) + w3·(1−S(p)) + w4·D(p), where w1, w2, w3, and w4 are weight coefficients used to balance the importance of different factors. L(p) is the path length, a positive number representing the total length of the path. The shorter the path, the smaller L(p). R(p) is the collision risk, a positive number representing the probability of collision along the path. The lower the collision risk, the smaller R(p). S(p) is grasping stability, a number between 0 and 1 that indicates the stability of the grasping point. The higher the stability, the closer S(p) is to 1. D(p) is dynamic performance, a positive number that indicates the rate of change of acceleration and velocity along the path. The better the dynamic performance, the smaller D(p).

[0075] Step S520 uses the grasping path cost function to perform path planning and cost evaluation within the material grasping path space, determining the target material grasping path with the lowest cost. Specifically, a path planning algorithm (such as the A* algorithm, the RRT algorithm, or the PRM algorithm) is used to search for paths within the material grasping path space. Each candidate path is evaluated using the grasping path cost function to calculate the total cost of the path. The path with the lowest cost is selected as the target material grasping path.

[0076] For example, suppose the RRT algorithm is used for path planning. The RRT algorithm searches for a path from the starting point to the end point in the material grasping path space through random sampling and tree growth. For each candidate path, its path length, collision risk, grasping stability, and dynamic performance are calculated, and the total cost is calculated based on the grasping path cost function. The path with the lowest total cost is selected as the target material grasping path. This implementation method gradually optimizes the generation process of the target material grasping path by constructing a grasping path cost function and using the grasping path cost function for path planning and cost evaluation. These technical means work together to improve the accuracy and reliability of path planning.

[0077] In one possible implementation, the 3D sorting machine performs material grabbing and compensation closed-loop control based on the target material grabbing point and target material grabbing force, as well as the target material grabbing path. Step S500 further includes step S530, in which the 3D sorting machine performs material grabbing feedback based on the target material grabbing point and target material grabbing force, as well as the target material grabbing path, to obtain material grabbing deviation parameters. Specifically, the 3D sorting machine controls the robotic arm to perform the grabbing operation based on the target material grabbing point, target material grabbing force and target material grabbing path. During the grabbing process, sensors (such as force sensors, visual sensors, position sensors) are used to monitor parameters such as the position, speed, and grabbing force of the robotic arm in real time. Through these sensor data, the deviation parameters in the grabbing process, such as position deviation, grabbing force deviation, posture deviation, etc., are calculated.

[0078] For example, a six-dimensional force sensor is installed at the end of the robotic arm to monitor the gripping force in real time. A binocular camera or structured light camera is used to monitor the position and posture of the material in real time. The robotic arm's joint encoders monitor the position and speed of the robotic arm in real time. The Euclidean distance between the actual gripping point and the target gripping point is calculated to determine the position deviation. The difference between the actual gripping force and the target gripping force is calculated to determine the gripping force deviation. The angular difference between the actual gripping posture and the target gripping posture is calculated to determine the posture deviation.

[0079] Step S540, based on the material grasping deviation parameters, the target material grasping force and the target material grasping path are compensated, adjusted, and closed-loop controlled. Specifically, according to the material grasping deviation parameters, the target material grasping force and the target material grasping path are adjusted. Use a PID controller or a fuzzy controller to adjust the grasping force and the motion trajectory of the robot arm in real time according to the material grasping deviation parameters. Monitor the deviation parameters in the grasping process in real time, and continuously adjust the grasping operation to ensure the accuracy and stability of the grasping. Closed-loop control can effectively reduce error accumulation and improve the success rate of grasping. This implementation method improves the accuracy and stability of the grasping operation of the 3D sorting machine through a real-time feedback mechanism and closed-loop control. By monitoring the deviation parameters in the grasping process in real time and performing compensation adjustments based on these parameters, the system can effectively reduce error accumulation and improve the success rate of grasping.

[0080] The embodiment of the present application uses a binocular structured light camera to collect multi-angle image data of the target material, and obtains a three-dimensional model of the material through three-dimensional reconstruction. Then, the material order data is read and matched and analyzed in combination with the three-dimensional model to obtain the material sorting grid position and material attribute information. Then, based on the working characteristic data of the 3D sorting machine, the three-dimensional model and material attribute information are grasped and analyzed to determine the grasping point and grasping force of the target material. Then, the material sorting grid position, the target material grasping point and the three-dimensional model are used as constraint parameters to construct a material grasping path space. Finally, the grasping path is planned within the space to generate the target material grasping path. The 3D sorting machine executes material grasping and implements technical means such as compensation closed-loop control based on this, which solves the technical problems of low control accuracy and reliability of the grasping control of the existing sorting machine, and achieves the technical effect of improving the accuracy and reliability of the grasping control of the sorting machine.

[0081] In the above, refer to Figure 1 A method for controlling the grabbing of a 3D sorting machine according to an embodiment of the present invention is described in detail. Figure 2 A grasping control system of a 3D sorting machine according to an embodiment of the present invention is described.

[0082] A 3D sorting machine grasping control system according to an embodiment of the present invention is designed to address the technical issues of low control accuracy and reliability in existing sorting machine grasping control, thereby improving the accuracy and reliability of the sorting machine's grasping control. The grasping control system includes a material three-dimensional model construction module 10, a matching analysis module 20, a grasping analysis module 30, a material grasping path space construction module 40, and a material grasping module 50.

[0083] The material three-dimensional model construction module 10 is used to acquire multi-angle image data of the target material through a binocular structured light camera, and perform three-dimensional reconstruction based on the multi-angle image data to obtain a three-dimensional model of the material; the matching analysis module 20 is used to read the material order data, perform matching analysis based on the material three-dimensional model and the material order data, and obtain the material sorting grid position and material material attribute information; the grasping analysis module 30 is used to grasp and analyze the material three-dimensional model and the material material attribute information based on the working characteristic data of the 3D sorting machine, and determine the target material grasping point and the target material grasping force; the material grasping path space construction module 40 is used to use the material sorting grid position, the target material grasping point and the material three-dimensional model as constraint parameters to construct the material grasping path space of the 3D sorting machine; the material grasping module 50 is used to perform grasping path planning in the material grasping path space to generate a target material grasping path, and the 3D sorting machine performs material grasping and compensation closed-loop control based on the target material grasping point and the target material grasping force, as well as the target material grasping path.

[0084] The specific configuration of the material 3D model construction module 10 will be described in detail below. As described above, to obtain a material 3D model, the material 3D model construction module 10 may further include: an available multi-angle image data acquisition unit for initializing a Gaussian filter to perform Gaussian filtering and contrast enhancement on the multi-angle image data to obtain available multi-angle image data; a multi-view registration depth map acquisition unit for performing structured light decoding and image registration on the available multi-angle image data to obtain a multi-view registration depth map; a global material point cloud data acquisition unit for loading the calibration parameters of the binocular structured light camera and the multi-view registration depth map for point cloud conversion and fusion to obtain global material point cloud data; and a material 3D model acquisition unit for performing gridding processing and 3D surface reconstruction based on the global material point cloud data to obtain a material 3D model.

[0085] Among them, the multi-view registration depth map is obtained, and the multi-view registration depth map acquisition unit can further include: a multi-angle absolute phase data acquisition subunit is used to perform phase calculation extraction and phase unwrapping on each image in the available multi-angle image data to obtain multi-angle absolute phase data; a depth conversion subunit is used to perform depth conversion on the multi-angle absolute phase data based on the calibration parameters of the binocular structured light camera to obtain multiple single-view depth maps; a feature point extraction subunit is used to extract feature points from each perspective image in the multiple single-view depth maps to obtain multiple image feature point sets; a registration and alignment subunit is used to use the multiple image feature point sets to perform registration and alignment on the multiple single-view depth maps to obtain the multi-view registration depth map.

[0086] Among them, the material three-dimensional model is obtained, and the material three-dimensional model acquisition unit can further include: a global point cloud normal information generation subunit is used to perform outlier filtering and normal estimation calculation on the global material point cloud data to generate global point cloud normal information; a structure division subunit is used to set Poisson reconstruction parameters, and perform structure division on the global material point cloud data based on the Poisson reconstruction parameters to construct a multi-level point cloud octree set; a Poisson equation solving subunit is used to calculate the vector field and solve the Poisson equation in the nodes of the multi-level point cloud octree set according to the global point cloud normal information to obtain an implicit function approximate solution; a three-dimensional surface reconstruction subunit is used to perform isosurface extraction and mesh post-processing based on the implicit function approximate solution to obtain the material three-dimensional model after three-dimensional surface reconstruction.

[0087] The specific configuration of the grasping and parsing module 30 will be described in detail below. As described above, to determine the target material grasping point and the target material grasping force, the grasping and parsing module 30 may further include: a surface feature recognition unit for performing surface feature recognition on the three-dimensional material model to obtain a set of candidate grasping points; a matching and optimization unit for performing matching and optimization on the set of candidate grasping points based on the working characteristic data of the 3D sorting machine to determine the target material grasping point; a center of gravity solving unit for performing center of gravity solution on the three-dimensional material model if the target material grasping point is zero to obtain material center of gravity information, and using the material center of gravity information as the target material grasping point; and a grasping and parsing unit for performing grasping and parsing on the three-dimensional material model and the material property information based on the working characteristic data of the 3D sorting machine to obtain the target material grasping force.

[0088] Among them, the grasping analysis unit for obtaining the target material grasping force may further include: a grasper restriction condition determination subunit for determining the grasper restriction condition based on the working characteristic data of the 3D sorting machine; a grasping force requirement decomposition subunit for decomposing the grasping force requirement of the material three-dimensional model and the material material attribute information to obtain the grasping gravity component, the grasping inertia component and the grasping safety component; a maximum value determination subunit for performing maximum value determination on the grasping gravity component, the grasping inertia component and the grasping safety component based on the grasper restriction condition to determine the grasping force of the target material.

[0089] The specific configuration of the material grabbing path space construction module 40 will be described in detail below. As described above, to construct the material grabbing path space of the 3D sorter, the material grabbing path space construction module 40 may further include: a grabbing path three-dimensional space initialization unit for initializing the grabbing path three-dimensional space based on the working range of the 3D sorter; and a constraint correction unit for constraining the grabbing path three-dimensional space using the material sorting grid position, the target material grabbing point, and the material three-dimensional model as constraint parameters to perform constraint correction on the grabbing path three-dimensional space to construct the material grabbing path space.

[0090] The specific configuration of the material grabbing module 50 will be described in detail below. As described above, to generate a target material grabbing path, the material grabbing module 50 may further include: a grabbing path cost function construction unit for constructing a grabbing path cost function based on the material grabbing planning objective; and a target material grabbing path determination unit for utilizing the grabbing path cost function to perform path planning and cost evaluation within the material grabbing path space, thereby determining the target material grabbing path with the minimum cost.

[0091] In which, the 3D sorting machine performs material grabbing and compensation closed-loop control based on the target material grabbing point and target material grabbing force, as well as the target material grabbing path, and the material grabbing module 50 may further include: a material grabbing feedback unit used for the 3D sorting machine to perform material grabbing feedback based on the target material grabbing point and target material grabbing force, as well as the target material grabbing path, to obtain material grabbing deviation parameters; a compensation adjustment unit used to perform compensation adjustment and closed-loop control on the target material grabbing force and the target material grabbing path based on the material grabbing deviation parameters.

[0092] A grasping control system for a 3D sorting machine provided in an embodiment of the present invention can execute a grasping control method for a 3D sorting machine provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0093] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules may be used and run on the user terminal and / or server, and the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other and are not used to limit the scope of protection of the present invention.

[0094] The above specific embodiments do not constitute a limitation to the scope of protection of this application. It should be understood by those skilled in the art that various modifications, combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of this application should be included in the scope of protection of this application. In some cases, the actions or steps recorded in this application can be performed in an order different from that in the embodiments and can still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A grasping control method for a 3D sorting machine, characterized in that: The method comprises: Acquire multi-angle image data of the target material by using a binocular structured light camera, and perform three-dimensional reconstruction based on the multi-angle image data to obtain a three-dimensional model of the material; Read the material order data, perform matching analysis based on the material three-dimensional model and the material order data, and obtain the material sorting grid position and material material attribute information; Based on the working characteristic data of the 3D sorter, the three-dimensional model of the material and the material attribute information of the material are grasped and analyzed to determine the target material grasping point and the target material grasping force; The material sorting grid position, the target material grabbing point and the material three-dimensional model are used as constraint parameters to construct the material grabbing path space of the 3D sorting machine; A grabbing path planning is performed in the material grabbing path space to generate a target material grabbing path. The 3D sorting machine performs material grabbing and compensation closed-loop control based on the target material grabbing point and target material grabbing force, as well as the target material grabbing path.

2. A grasping control method for a 3D sorting machine according to claim 1, characterized in that: The method of obtaining a three-dimensional model of the material includes: Initializing a Gaussian filter to perform Gaussian filtering and contrast enhancement on the multi-angle image data to obtain usable multi-angle image data; Performing structured light decoding and image registration on the available multi-angle image data to obtain a multi-view registered depth map; Loading the calibration parameters of the binocular structured light camera and the multi-view registration depth map to perform point cloud conversion and fusion to obtain global material point cloud data; Grid processing and three-dimensional surface reconstruction are performed based on the global material point cloud data to obtain a three-dimensional model of the material.

3. A grabbing control method for a 3D sorting machine according to claim 2, characterized in that: The obtaining of a multi-view registration depth map includes: Performing phase calculation extraction and phase unwrapping on each image in the available multi-angle image data to obtain multi-angle absolute phase data; Performing depth conversion on the multi-angle absolute phase data based on calibration parameters of the binocular structured light camera to obtain multiple single-view depth maps; Extracting feature points from each perspective image in the multiple single-perspective depth maps to obtain multiple image feature point sets; The multiple single-view depth maps are registered and aligned using the multiple image feature point sets to obtain the multi-view registered depth map.

4. The grasping control method of a 3D sorting machine according to claim 2, characterized in that: The obtaining of the three-dimensional model of the material comprises: Performing outlier filtering and normal estimation calculation on the global material point cloud data to generate global point cloud normal information; Setting Poisson reconstruction parameters, structurally dividing the global material point cloud data based on the Poisson reconstruction parameters, and constructing a multi-level point cloud octree set; Calculating a vector field in the nodes of the multi-level point cloud octree set according to the global point cloud normal information and solving the Poisson equation to obtain an implicit function approximate solution; Based on the implicit function approximate solution, isosurface extraction and mesh post-processing are performed to obtain the three-dimensional model of the material after three-dimensional surface reconstruction.

5. The grasping control method of a 3D sorting machine according to claim 1, characterized in that: The determining of the target material grasping point and the target material grasping force includes: Performing surface feature recognition on the three-dimensional model of the material to obtain a set of candidate grasping points; Match and optimize the candidate grabbing point set based on the working characteristic data of the 3D sorting machine to determine the target material grabbing point; If the target material grabbing point is zero, the center of gravity of the three-dimensional model of the material is solved to obtain the material center of gravity information, and the material center of gravity information is used as the target material grabbing point; The three-dimensional model of the material and the material property information of the material are grasped and analyzed based on the working characteristic data of the 3D sorting machine to obtain the target material grasping force.

6. A grasping control method for a 3D sorting machine according to claim 5, characterized in that: The method of obtaining the target material gripping force comprises: determining gripper restriction conditions based on the operating characteristic data of the 3D sorting machine; Decomposing the grasping force requirement of the material three-dimensional model and the material property information of the material to obtain a grasping gravity component, a grasping inertia component, and a grasping safety component; The maximum values ​​of the grasping gravity component, the grasping inertia component and the grasping safety component are determined based on the grasper restriction conditions to determine the target material grasping force.

7. The grasping control method of a 3D sorting machine according to claim 1, characterized in that: The step of constructing the material grabbing path space of the 3D sorting machine includes: Initialize the three-dimensional space of the grasping path according to the working range of the 3D sorting machine; The material sorting grid position, the target material grabbing point and the material three-dimensional model are used as constraint parameters to perform constraint correction on the grabbing path three-dimensional space to construct the material grabbing path space.

8. The grasping control method of a 3D sorting machine according to claim 1, characterized in that: The generating of the target material grabbing path includes: Construct a grasping path cost function based on the material grasping planning goal; The grasping path cost function is used to perform path planning and cost evaluation in the material grasping path space to determine the target material grasping path with the minimum cost.

9. The grasping control method of a 3D sorting machine according to claim 1, characterized in that: The 3D sorting machine performs material grabbing and compensation closed-loop control based on the target material grabbing point, the target material grabbing force, and the target material grabbing path, including: The 3D sorting machine performs material grabbing feedback based on the target material grabbing point and the target material grabbing force, as well as the target material grabbing path, to obtain a material grabbing deviation parameter; The target material grabbing force and the target material grabbing path are compensated, adjusted, and closed-loop controlled based on the material grabbing deviation parameter.

10. A grasping control system for a 3D sorting machine, characterized in that: The system is used to implement the grasping control method of a 3D sorting machine according to any one of claims 1 to 9, and the system includes: A material three-dimensional model construction module is used to acquire multi-angle image data of the target material through a binocular structured light camera, and perform three-dimensional reconstruction based on the multi-angle image data to obtain a three-dimensional model of the material; A matching analysis module is used to read the material order data, perform matching analysis based on the material three-dimensional model and the material order data, and obtain the material sorting grid position and material material attribute information; A grasping and analyzing module is used to grasp and analyze the three-dimensional model of the material and the material property information of the material based on the working characteristic data of the 3D sorting machine, and determine the target material grasping point and the target material grasping force; A material grabbing path space construction module is used to construct the material grabbing path space of the 3D sorting machine by taking the material sorting grid position, the target material grabbing point and the material three-dimensional model as constraint parameters; The material grabbing module is used to plan the grabbing path in the material grabbing path space and generate a target material grabbing path. The 3D sorting machine performs material grabbing and compensation closed-loop control based on the target material grabbing point and target material grabbing force, as well as the target material grabbing path.

Images