Object recognition and pose estimation method for grabbing robot

Abstract

The invention discloses an object recognition and pose estimation method for a grabbing robot, and relates to the technical field of robot automatic grabbing, and the method comprises the following steps: obtaining multi-modal data through simulating a visual array of insect compound eyes and an auditory sensor for bat echo positioning, and obtaining a fusion model through deep learning feature fusion network processing; an object recognition and grabbing strategy model is constructed based on reinforcement learning, the former combines multi-factor training and optimal selection, and the latter determines a strategy according to a large number of tests and can be dynamically adjusted. A specific algorithm is combined with SIFT and deep learning to calculate a pose, and fine tuning is carried out by means of physical simulation, particle swarm optimization and a force sensor; according to the method, the accuracy, stability and intelligence of recognition, pose estimation and grabbing of the robot in a complex environment are remarkably improved, automatic operation in the fields of industrial production, logistics storage and the like is facilitated, and the method has wide application prospects.

Description

Technical Field

[0001] The present invention relates to the technical field of robot automated grasping, and in particular to an object recognition and posture estimation method of a grasping robot. Background Art

[0002] With the rapid development of modern industrial automation and intelligent logistics, gripping robots have been widely used in many fields. On industrial production lines, gripping robots are required to accurately identify and grasp various parts to complete complex assembly tasks; in logistics warehousing, a large number of goods of different shapes, sizes and materials need to be quickly sorted and moved. However, existing gripping robot technology still faces many challenges.

[0003] Traditional grasping robots mostly rely on a single visual sensor, such as images or point cloud data obtained by ordinary industrial cameras or depth cameras, to perform object recognition and pose estimation. This single-modality perception method has limitations. In complex environments, such as when the lighting changes dramatically, there are occlusions, or the surface textures of objects are similar, it is difficult to accurately recognize objects and determine their poses based solely on visual information, which can easily lead to grasping failures or inaccurate grasping positions, thus affecting production efficiency and logistics automation processes.

[0004] In view of this, this application is hereby filed. Summary of the invention

[0005] The purpose of the present invention is to provide an object recognition and posture estimation method for a grasping robot, so as to solve the problem of poor object recognition accuracy and adaptability of the existing grasping robot in complex environments mentioned in the above background technology.

[0006] In order to solve the above technical problems, the present invention provides an object recognition and posture estimation method for a grasping robot, comprising the following steps: S1: multimodal perception data acquisition: a visual sensor array simulating the structure of insect compound eyes is used to collect multi-perspective image data of the environment, and an auditory sensor based on the principle of bat echolocation is used to obtain acoustic feature data and distance information of the object. Each sub-sensor of the visual sensor array has an adaptive light adjustment function to adapt to different lighting environments; S2: fusion perception data processing: the visual image data and the auditory data are fused, and a feature fusion network based on deep learning is used. The feature fusion network first encodes the features of the visual and auditory data respectively, and then fuses the features through a fully connected layer, thereby constructing a fusion perception feature model; S3: object recognition model training: an object recognition model is constructed based on reinforcement learning, in which the action space covers a combination of multiple deep learning architectures and traditional feature extraction methods, and the reward function comprehensively considers the recognition accuracy, recognition speed and resource consumption. The robot is trained on a variety of object data sets to determine the optimal recognition strategy; S4: preliminary posture estimation :Based on the multi-view images of the visual sensor array, the preliminary pose information of the object is calculated by using a stereo vision algorithm based on feature point matching combined with an optical flow algorithm based on motion estimation. The feature point matching uses a combination of SIFT algorithm and deep learning feature matching to improve the matching accuracy; S5: Pose optimization and verification: A physical simulation model is constructed based on the material properties, shape information of the object and the mechanical parameters of the robot's grasping components. The preliminary pose information is input into the model for grasping simulation. The pose is optimized using a particle swarm optimization algorithm. The optimization goal is the minimum stress and maximum stability during the grasping process. At the same time, the force sensor is used to verify and fine-tune the optimized pose in the actual scene; S6: Grasping strategy generation and execution: With the help of reinforcement learning, a grasping strategy model is designed. The action space includes grasping point selection, grasping force control and grasping path planning. The reward function is constructed based on the grasping success rate, the degree of damage to the object and the grasping time. The robot obtains the optimal grasping strategy through a large number of simulated grasping and actual experimental learning. When performing grasping tasks, the grasping strategy is dynamically adjusted according to the real-time perceived object pose and environmental changes.

[0007] Furthermore, in S1, the viewing angle range of the sub-sensors of the visual sensor array is between 30°-60°, and the overlapping area of ​​the viewing angles of adjacent sub-sensors does not exceed 20%; the sound wave transmission frequency of the auditory sensor is adjustable within the range of 30kHz-70kHz, and the receiving sensitivity is not less than -50dB.

[0008] Furthermore, in S2, the feature fusion network of deep learning adopts a residual network structure, and the network depth is 10-20 layers.

[0009] Furthermore, in S3, in the object recognition model training step, reinforcement learning adopts the A3C algorithm, and the number of parallel training threads is 4-8.

[0010] Furthermore, in S4, the feature point matching threshold of the stereoscopic vision algorithm based on feature point matching is set between 0.01-0.05.

[0011] Furthermore, in S5, the physical simulation model is constructed using multi-body dynamics software, and the time step of the model is set to 0.001-0.01 seconds.

[0012] Furthermore, in S5, the number of particles based on the particle swarm optimization algorithm is 20-50, and the inertia weight decreases linearly from 0.9 to 0.4.

[0013] Furthermore, in S6, the value function of the reinforcement learning is approximated by a neural network, and the activation function of the neural network is a ReLU function.

[0014] Furthermore, it also includes: a data storage strategy, the data storage strategy adopts a distributed storage system, and the data reading speed is not less than 500MB / s.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention adopts multimodal perception, integrates visual and auditory features, and makes up for the limitation of single vision. The feature fusion network based on deep learning deeply processes multimodal data, and builds an object recognition model with the reinforcement learning A3C algorithm. Its action space is diverse, and it can automatically select the best object, accurately identify various objects, and has strong adaptability; improve the accuracy and adaptability of object recognition; 2. Use stereo vision and optical flow algorithms combined with SIFT and deep learning feature matching to calculate the initial pose and improve accuracy. Build a physical simulation model, use particle swarm optimization and fine-tune with force sensor feedback, consider physics and actual grasping, reduce contact stress, and accurately and stably grasp objects such as center of gravity offset; optimize pose estimation accuracy and stability; 3. Design a grasping strategy model by using reinforcement learning, and determine the optimal strategy through a large number of simulations and actual experiments, including grasping points, strength, path planning, etc. During execution, it is dynamically adjusted according to the object posture and environment, such as using a sampling path planning algorithm to replan the path, improve the grasping success rate, reduce the risk of damage, and be intelligent and flexible, enhancing the intelligence and flexibility of the grasping strategy; 4. The reward function of the object recognition model training integrates multiple factors to encourage the robot to maintain accuracy while increasing speed and reducing resource consumption. This not only increases the amount of grasping tasks and improves work efficiency, but also makes efficient use of resources and reduces costs, providing a cost-effective solution for industrial production and logistics warehousing, and improving overall system efficiency and resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a block diagram of the principle of an object recognition and pose estimation method for a grasping robot. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] See also Figure 1 The present invention provides a technical solution: a method for object recognition and posture estimation of a grasping robot, comprising the following steps: Implementation environment construction: This embodiment is tested in an indoor logistics sorting warehouse with an area of ​​about 500 square meters. Thousands of objects of various shapes, sizes and materials are placed in the warehouse, distributed on shelves of different heights (between 0.5 meters and 2 meters) and arranged relatively densely, simulating a real and complex logistics sorting scenario.

[0019] The grasping robot is equipped with a unique visual sensor array that simulates the structure of insect compound eyes. It is carefully installed on the 360-degree rotating pan-tilt on the top of the robot to obtain environmental images in all directions without blind spots. The visual sensor array consists of 10 sub-sensors, and the viewing angle range of each sub-sensor is precisely set to 45°. The overlapping area of ​​adjacent sub-sensors is strictly controlled at 15%. This design not only ensures the integrity of the field of view, but also avoids excessive information redundancy. The sub-sensors have highly sensitive adaptive light adjustment functions. Whether in dim corners (about 10lux) or in areas of direct strong light (about 1000lux), they can quickly and automatically adjust parameters such as aperture and shutter speed to ensure the clarity and accuracy of the collected images. For example, in dim light, the sub-sensor will automatically increase the aperture and reduce the shutter speed to obtain enough light input; in strong light, it will reduce the aperture and increase the shutter speed accordingly to prevent overexposure of the image.

[0020] At the same time, the robot is also equipped with advanced auditory sensors designed based on the principle of bat echolocation. Its sound wave transmission frequency can be flexibly adjusted within the range of 40kHz-60kHz, and the receiving sensitivity is as high as -45dB. It is cleverly installed on both sides of the robot body to assist in accurately detecting the position of objects and obtaining rich acoustic features. This multi-modal sensor configuration provides the robot with the ability to fully perceive the environment.

[0021] The robot's core computing system uses a powerful server equipped with multiple high-end GPUs, which is specially designed to efficiently run complex deep learning models and various data processing algorithms. Data storage relies on a distributed storage system with an excellent data reading speed of up to 600MB / s, which can quickly respond to the robot's storage and reading needs for large amounts of images and data during operation, ensuring the smoothness and efficiency of the entire crawling process.

[0022] A method for object recognition and position estimation of a grasping robot, comprising: S1: Multimodal perception data acquisition: Visual data collection: When the robot receives the grasping task instruction from the central control system, it immediately starts the visual sensor array to collect environmental images. For example, when faced with a pile of packages of various shapes and placed in a disorderly manner, the 10 sub-sensors of the visual sensor array simultaneously capture images from multiple angles, stably collecting 30 frames of high-resolution (1920×1080 pixels) image data per second, which comprehensively covers the rich and detailed information such as the color, texture, and shape of the object. The intelligent light adjustment system inside each sub-sensor automatically adjusts key parameters such as aperture size and shutter speed according to the current ambient light intensity. Specifically, when the light intensity is between 10-50lux, the aperture will gradually increase and the shutter speed will be appropriately slowed down; when the light intensity is between 500-1000lux, the aperture will gradually decrease and the shutter speed will be accelerated, so as to ensure that the image is always clear and sharp, providing a high-quality visual data foundation for subsequent object recognition and pose estimation.

[0023] Auditory data collection: At the same time, the auditory sensor quickly emits a specific sound wave with a frequency of 50kHz. After encountering an object, the sound wave is reflected back and accurately received by the sensor. By accurately analyzing the time difference between the emission and reception of the sound wave (accurate to the nanosecond level) and the frequency change (resolution of 0.1kHz), the distance information between the object and the robot can be accurately determined. For example, for a rectangular package 2 meters away from the robot, the auditory sensor can accurately measure its distance error within the range of ±0.05 meters. In addition, by performing complex spectrum analysis algorithms on the reflected sound wave signals, such as using high-precision fast Fourier transform (FFT), the acoustic characteristics of the reflected sound waves of the package surface material are deeply obtained, including the unique acoustic signal characteristics corresponding to the hardness, roughness and other information of the material. These characteristics will serve as important auxiliary information for object recognition and posture estimation, further improving the robot's perception accuracy of objects.

[0024] A method for object recognition and position estimation of a grasping robot, comprising: S2: Fusion perception data processing: The collected massive visual image data and fine auditory data are input into the feature fusion network based on deep learning. The network adopts a carefully designed 15-layer residual network structure and has powerful feature extraction and fusion capabilities.

[0025] First, the visual data enters the convolution layer for deep feature encoding. Each pixel of the input image is scanned and operated by a series of convolution kernels. For example, a convolution kernel of size 3×3 is used to perform a convolution operation on the image. By continuously adjusting the weight parameters of the convolution kernel, key feature maps such as edges, corners, and textures are extracted. After multiple layers of convolution and pooling operations, the visual data is mapped to a low-dimensional feature space for subsequent fusion with auditory features.

[0026] Auditory data is processed by a specially developed acoustic feature extraction module. This module performs a discrete Fourier transform (DFT) on the sound wave signal sequence, converting the time domain signal into a frequency domain signal, thereby clearly extracting important features such as the frequency and amplitude changes of the sound wave. For example, for a sound wave signal reflected by an object of a specific material, the DFT can accurately obtain its energy distribution at different frequencies, and these feature information is also mapped to a low-dimensional feature space after processing.

[0027] Finally, another fully connected layer is used to organically fuse the low-dimensional features of vision and hearing to obtain a fused perception feature model. When processing a batch of perception data containing 100 different objects, the network can quickly complete feature fusion processing within 5 seconds with its efficient architecture and powerful computing power, accurately obtain the fused feature representation of each object, and provide solid data support for subsequent object recognition.

[0028] A method for object recognition and position estimation of a grasping robot, comprising: S3: Object recognition model training: The A3C algorithm based on reinforcement learning builds a highly intelligent object recognition model. The action space covers a diverse combination of 5 carefully selected deep learning architectures (such as convolutional neural networks with different numbers of layers and parameter configurations) and 3 traditional and effective feature extraction methods (such as SIFT, HOG, and LBP). A total of 15 different recognition strategies can be flexibly selected by the robot according to actual conditions.

[0029] The reward function has been carefully designed and repeatedly optimized, taking into account several key factors: for every 1% increase in recognition accuracy, the reward increases by 10 points, which encourages the robot to continuously improve recognition accuracy; for every 1 second increase in recognition speed, the reward increases by 5 points, prompting the robot to complete the recognition task as quickly as possible while ensuring accuracy; for every 1% decrease in resource consumption (such as GPU memory usage and computing time), the reward increases by 3 points to ensure that the robot uses computing resources efficiently during operation and avoids resource waste. For example, if the robot's recognition accuracy increases from 90% to 91%, the recognition time is shortened from 3 seconds to 2.5 seconds, and the resource consumption is reduced by 2% in a recognition task, the corresponding reward increase score can be calculated according to the reward function, thereby guiding the robot to continuously optimize its recognition strategy.

[0030] The robot was trained for a long time on a large-scale, diverse data set containing 10,000 different objects, using 6 parallel training threads for simultaneous training, with each thread processing a different subset of training data, making full use of computing resources to improve training efficiency. After 1,000 training cycles of repeated learning and optimization, the robot was able to accurately determine the optimal recognition strategy based on the characteristics of different types of objects. For example, for objects with extremely rich and complex surface textures, the robot will intelligently choose a strategy based on a combination of a specific convolutional neural network architecture and SIFT feature extraction, giving full play to the advantages of both in processing texture features; for objects with regular shapes but single colors, another shallower convolutional neural network and LBP feature extraction combination strategy will be used to improve recognition efficiency and reduce resource consumption.

[0031] A method for object recognition and position estimation of a grasping robot, comprising: S4: Preliminary estimation of posture: Based on the multi-view images collected by the visual sensor array, the preliminary position information of the object is calculated using a stereo vision algorithm based on feature point matching and an optical flow algorithm based on motion estimation.

[0032] In the feature point matching link, the SIFT algorithm is innovatively combined with deep learning feature matching. The SIFT algorithm can stably extract local feature points in the image, while the deep learning feature matching uses the key feature points of the object learned by the pre-trained neural network. The two complement each other to improve the accuracy and reliability of matching. The feature point matching threshold is set to 0.03 after a lot of experiments and optimization. For example, when processing an image of an object with a complex shape, the SIFT algorithm first extracts multiple feature points on the surface of the object, such as the corners of the edge, the mutation points of the texture, etc. At the same time, the deep learning feature matching network quickly locates the corresponding key feature points based on the category information of the object, and then accurately calculates the similarity between the two. When the similarity exceeds the threshold of 0.03, the feature point matching is considered successful. By matching multiple feature points and combining the principle of stereoscopic vision, the preliminary position coordinates of the object in three-dimensional space are calculated.

[0033] For the stereo vision algorithm, the depth information of the object is calculated by accurately measuring the difference between the coordinates of the feature points on the left camera image and the corresponding feature points on the right camera image, as well as the known camera baseline length (e.g. 0.5 meters) and focal length (e.g. 0.05 meters). For example, for an object located at a specific position, the depth value of the object from the robot can be accurately obtained by calculating the parallax of the feature points in the left and right camera images, and then combined with the camera's intrinsic parameter matrix, the preliminary position coordinates of the object in three-dimensional space can be determined.

[0034] The optical flow algorithm further refines the rotation and translation information of the object by deeply analyzing the motion characteristics of the object in continuous image frames. By accurately tracking the coordinate changes of feature points in the image sequence between different frames, for example, in a 10-frame image sequence, a feature point moves from the coordinates (100, 200) of the first frame to the coordinates (110, 205) of the tenth frame, the optical flow vector is calculated based on these coordinate changes, and then the least squares method is used to fit the motion model of the object. The initial estimate of the object's posture information error is within the range of ±5° rotation angle error and ±0.1 meter translation error, providing a more accurate initial value for subsequent posture optimization.

[0035] A method for object recognition and position estimation of a grasping robot, comprising: S5: Pose optimization and verification: Based on the material properties of the object (such as plastic, metal, paper and other common materials), shape information (such as cubes, spheres, cylinders and other basic shapes and complex combination shapes) and precise mechanical parameters of the robot grasping parts (mechanical grippers with a gripping force adjustment range of 20N-80N and an angle adjustment range of 0-180 degrees), a highly realistic physical simulation model is constructed using professional multi-body dynamics software, and the time step is precisely set to 0.005 seconds to ensure the accuracy and reliability of the simulation.

[0036] The preliminarily estimated posture information is input into the physical simulation model for comprehensive grasping simulation, and the advanced particle swarm optimization algorithm is used to deeply optimize the posture. The number of particles is set to 30, and the inertia weight is gradually reduced from 0.9 to 0.4 according to a specific linear decreasing law. During the optimization process, each particle represents a possible posture solution, and the position and velocity of the particles are continuously updated iteratively to find the optimal posture solution. For example, when dealing with an object with an offset center of gravity and an irregular shape, the particle swarm optimization algorithm gradually adjusts the position of the particles (i.e. the posture of the object) during the iteration process according to the physical characteristics of the object and the grasping requirements, so that the contact stress between the mechanical gripper and the object is minimized and the grasping stability is maximized during the grasping process.

[0037] In actual scenarios, the optimized posture is verified and fine-tuned in real time using high-precision force sensors installed on the mechanical gripper. When the robot attempts to grasp an object with a complex surface shape, the force sensor monitors the force changes during the grasping process in real time at an extremely high frequency (e.g. 1000Hz). If the force distribution is uneven or abnormal fluctuations occur (e.g. the force difference exceeds 5N), the posture is fine-tuned according to the pre-set feedback adjustment strategy. The adjustment range is strictly controlled within the translation range of ±0.05 meters and the rotation range of ±2°. This precise fine-tuning mechanism further improves the success rate and stability of grasping.

[0038] A method for object recognition and position estimation of a grasping robot, comprising: S6: Crawl strategy generation and execution: With the help of reinforcement learning, an intelligent grasping strategy model is designed. The action space includes several key elements such as grasping point selection, grasping force control, and grasping path planning.

[0039] The reward function is carefully constructed based on the grasping success rate (the reward increases by 20 points for each successful grasp), the degree of damage to the object (the reward decreases by 10 points if minor damage is caused, and by 30 points if severe damage is caused), and the grasping time (the reward increases by 8 points for every 1 second shortened). For example, in a grasping task, if the robot successfully grasps an object without causing any damage to it, and the grasping time is 2 seconds, the corresponding reward score can be obtained according to the reward function, which encourages the robot to continuously optimize the grasping strategy.

[0040] The robot continuously learns and accumulates experience by conducting 5,000 simulated grasping tests and 1,000 actual grasping tests in a simulated environment, and gradually determines the optimal grasping strategy. For example, for small objects with smooth surfaces, after a large number of experimental learning, the robot will choose a position near the center of gravity of the object and where the surface friction is relatively large as the grasping point, while accurately controlling the grasping force within a moderate range, and the grasping path uses a smooth curve to cleverly avoid surrounding obstacles; for large and heavy objects, it will reasonably select multiple grasping points based on the shape and weight distribution characteristics of the object, and accurately allocate the grasping force of each grasping point. The grasping path planning will fully consider the complexity of the surrounding environment to ensure that the grasping task is completed safely and efficiently.

[0041] When performing grasping tasks, the robot dynamically adjusts the grasping strategy based on the real-time perception of the object's posture and environmental changes. When new obstacles suddenly appear in the environment or the object's position moves slightly due to external interference (for example, the moving distance is within 0.1 meters), the robot can quickly use sampling-based path planning algorithms such as the Rapid Exploration Random Tree (RRT) algorithm to re-plan a safe and efficient grasping path in a very short time (for example, within 0.5 seconds). At the same time, the grasping point and grasping force are adjusted in real time according to the changes in the object's posture to ensure the smooth completion of the grasping task, greatly improving the robot's grasping adaptability and reliability in complex environments.

[0042] In summary, the existing technologies are often limited to single visual modality perception, fixed recognition algorithms, and posture estimation and grasping strategies that lack full consideration of physical characteristics. The present invention innovatively introduces multimodal perception fusion, uses a visual sensor array that simulates the structure of insect compound eyes and an auditory sensor based on the principle of bat echolocation to work together, and uses a deep learning network for deep fusion processing; builds a flexible and adaptive object recognition model and grasping strategy model based on reinforcement learning; and uses a posture optimization mechanism based on physical simulation and particle swarm optimization algorithm combined with force sensor feedback to achieve more accurate recognition of objects in complex environments, more precise and stable posture estimation, and more intelligent and flexible grasping operations. These significant distinguishing features bring about many beneficial effects, including greatly improving the accuracy and adaptability of object recognition, significantly optimizing the accuracy and stability of posture estimation, greatly enhancing the intelligence and flexibility of grasping strategies, and effectively improving the efficiency and resource utilization of the overall system. The present invention provides a more powerful, reliable and efficient solution for the widespread application of grasping robot technology in many fields such as industrial production, logistics warehousing, etc., which has strongly promoted the further development and innovation of robot automated grasping technology, and has broad market application prospects and extremely high technical value.

Claims

1. A method for object recognition and pose estimation of a grasping robot, characterized in that: The following steps are involved: S1: Multimodal perception data acquisition: The visual sensor array that simulates the structure of insect compound eyes collects multi-view image data of the environment, and uses auditory sensors based on the principle of bat echolocation to obtain acoustic feature data and distance information of objects. Each sub-sensor of the visual sensor array has an adaptive light adjustment function to adapt to different lighting environments; S2: Fusion perception data processing: Fusion of visual image data and auditory data, using a feature fusion network based on deep learning. The feature fusion network first encodes the features of visual and auditory data respectively, and then fuses the features through a fully connected layer to build a fusion perception feature model. S3: Object recognition model training: Build an object recognition model based on reinforcement learning, where the action space covers a combination of multiple deep learning architectures and traditional feature extraction methods. The reward function comprehensively considers recognition accuracy, recognition speed, and resource consumption. The robot is trained on a variety of object data sets to determine the optimal recognition strategy; S4: Preliminary pose estimation: Based on the multi-view images of the visual sensor array, the stereo vision algorithm based on feature point matching is combined with the optical flow algorithm based on motion estimation to calculate the preliminary pose information of the object. The feature point matching uses the SIFT algorithm combined with deep learning feature matching to improve the matching accuracy. S5: Posture optimization and verification: A physical simulation model is constructed based on the material properties and shape information of the object and the mechanical parameters of the robot grasping parts. The preliminary posture information is input into the model for grasping simulation. The posture is optimized using a particle swarm optimization algorithm. The optimization goal is to minimize stress and maximize stability during the grasping process. At the same time, the optimized posture is verified and fine-tuned using force sensors in actual scenarios. S6: Grasping strategy generation and execution: The grasping strategy model is designed with the help of reinforcement learning. The action space includes grasping point selection, grasping force control and grasping path planning. The reward function is constructed based on the grasping success rate, the degree of damage to the object and the grasping time. The robot obtains the optimal grasping strategy through a large number of simulated grasping and actual experimental learning. When executing grasping tasks, the grasping strategy is dynamically adjusted according to the real-time perception of the object posture and environmental changes.

2. The object recognition and pose estimation method of a grasping robot according to claim 1, characterized in that: In S1, the viewing angle range of the sub-sensors of the visual sensor array is between 30° and 60°, and the overlapping area of ​​the viewing angles of adjacent sub-sensors does not exceed 20%; the sound wave transmission frequency of the auditory sensor is adjustable within the range of 30kHz-70kHz, and the receiving sensitivity is not less than -50dB.

3. The object recognition and pose estimation method of a grasping robot according to claim 1, characterized in that: In S2, the feature fusion network of deep learning adopts a residual network structure, and the network depth is 10-20 layers.

4. The object recognition and pose estimation method of a grasping robot according to claim 1, characterized in that: In S3, in the object recognition model training step, reinforcement learning adopts the A3C algorithm, and the number of parallel training threads is 4-8.

5. The object recognition and pose estimation method of a grasping robot according to claim 1, characterized in that: In S4, the feature point matching threshold of the stereo vision algorithm based on feature point matching is set between 0.01-0.

05.

6. The object recognition and pose estimation method of a grasping robot according to claim 1, characterized in that: In S5, the physical simulation model is constructed using multi-body dynamics software, and the time step of the model is set to 0.001-0.01 seconds.

7. The object recognition and pose estimation method of a grasping robot according to claim 1, characterized in that: In S5, the number of particles based on the particle swarm optimization algorithm is 20-50, and the inertia weight decreases linearly from 0.9 to 0.

4.

8. The object recognition and pose estimation method of a grasping robot according to claim 1, characterized in that: In S6, the value function of the reinforcement learning is approximated by a neural network, and the activation function of the neural network is a ReLU function.

9. An object recognition and pose estimation method for a grasping robot according to any one of claims 1 to 8, characterized in that: Also includes: Data storage strategy,The data storage strategy adopts a distributed storage system, and the data reading speed is not less than 500MB / s.

Images