Robot positioning and grabbing system and method based on laser visual guidance

Abstract

The invention provides a robot positioning and grabbing system and method based on laser visual guidance, and relates to the technical field of robots, and the method comprises the steps that an intelligent processing module receives information output by a multi-mode sensing module and carries out intelligent processing operation; the intelligent processing operation comprises an image enhancement operation, a dynamic attention modeling operation and an online learning operation; the self-adaptive control module comprises a mixed pose control unit and a predictive maintenance unit; and the mixed pose control unit adopts a robust inverse kinematics solution based on Lie group and a force-position mixed control mode automatic switching algorithm to calculate the joint angle of the robot and the pose of an end effector. Rich environment and object information is acquired through the multi-modal sensing module, and image enhancement, dynamic attention modeling and online learning operation of the intelligent processing module are combined, so that the recognition precision of a target object is improved, and the influence of illumination and object surface characteristics on recognition is effectively reduced.

Description

Technical Field

[0001] The present invention relates to the field of robot technology, and in particular to a robot positioning and grasping system and method based on laser vision guidance. Background Art

[0002] As industrial production continues to increase its requirements for automation and precision, traditional robot grasping methods are difficult to meet complex scenarios and high-precision requirements. Laser vision guidance technology has emerged to provide a more accurate and efficient solution for robot positioning and grasping. Stereoscopic vision methods in common visual measurement technologies, such as binocular vision and laser line scanning technology, have been widely used in robot positioning to effectively ensure grasping accuracy.

[0003] However, the existing robot positioning and grasping systems have many shortcomings. For example, in terms of visual recognition, the traditional system is greatly affected by factors such as lighting and surface characteristics of objects, making it difficult to accurately identify the target object. In terms of motion control, it is difficult to achieve stable grasping of objects of different shapes and materials. At the same time, there is a lack of effective maintenance mechanism. As the use time increases, the wear of the robot arm joints will lead to a decrease in grasping accuracy. These problems limit the application and work efficiency of robots in complex environments. Therefore, it is necessary to provide a robot positioning and grasping system and method based on laser vision guidance to solve the above technical problems. Summary of the invention

[0004] The purpose of the present invention is to provide a robot positioning and grasping system and method based on laser vision guidance, so as to solve the deficiencies of the existing robot positioning and grasping systems in visual recognition, motion control and maintenance mechanism, realize accurate positioning and stable grasping of target objects, and improve the working efficiency and grasping accuracy of the robot in complex environments.

[0005] In order to solve the above technical problems, the present invention provides a robot positioning and grasping system and method based on laser vision guidance, including a multimodal perception module, an intelligent processing module, an adaptive control module and a mechanical execution module;

[0006] The multimodal perception module is used to obtain the three-dimensional structure, material spectrum, environmental information, visual image and tactile information of the object;

[0007] The intelligent processing module receives the information output by the multimodal perception module and performs intelligent processing operations; the intelligent processing operations include image enhancement operations, dynamic attention modeling operations and online learning operations;

[0008] The adaptive control module includes a hybrid posture control unit and a predictive maintenance unit;

[0009] The hybrid posture control unit uses a robust kinematics inverse solution based on Lie groups and an automatic switching algorithm of the force-position hybrid control mode to calculate the joint angles and the end effector posture of the robot and generate corresponding control instructions;

[0010] The predictive maintenance unit monitors the state of the robot arm joints in real time, and adjusts the control parameters of the robot through the robot arm joint wear prediction model and the control parameter online optimization algorithm;

[0011] The mechanical execution module receives the instructions issued by the adaptive control module and executes the positioning and grasping operation of the object; wherein the instructions issued by the adaptive control module include posture control instructions, force-position mixed control instructions and control parameter adjustment instructions.

[0012] Preferably, the multimodal perception module includes a multispectral laser scanner, an environmental parameter sensor, a bionic compound eye camera and a tactile feedback gripper;

[0013] The multispectral laser scanner adjusts the wavelength through the wavelength dynamic adjustment module according to the material characteristics of the object to be identified and the ambient light conditions, scans the object, and obtains the three-dimensional structure and material spectrum information of the object;

[0014] Environmental parameter sensors are used to monitor ambient temperature, humidity, and dust concentration in real time;

[0015] The bionic compound eye camera uses a multi-channel fisheye lens array for panoramic dynamic visual acquisition, and the dynamic aperture adjustment unit automatically adjusts the aperture size according to changes in ambient light; the image preprocessing chip is used to perform preliminary preprocessing operations on the acquired images, including image noise reduction and enhancement;

[0016] When the tactile feedback gripper contacts an object, a micro pressure sensor matrix and a vibration tactile feedback module provide tactile information.

[0017] Preferably, the image enhancement operation specifically uses a generative adversarial network to optimize image quality; the adversarial network includes a generator and a discriminator;

[0018] The low-light multispectral image collected by the multimodal perception module is input into the generator G of the generative adversarial network, and multi-level feature extraction and reconstruction are performed through the convolution layer C, pooling layer P and deconvolution layer Dc to generate an enhanced image; the discriminator D distinguishes the enhanced image from the real high-quality image, and adjusts the parameters of the generator and discriminator through continuous iteration until the image reaches the convergence condition.

[0019] Preferably, the dynamic attention modeling operation specifically adopts a spatiotemporal attention mechanism; the spatiotemporal attention mechanism is composed of a spatial attention unit and a temporal attention unit working together;

[0020] The spatial attention module processes the enhanced image, calculates the channel attention and the spatial attention, and combines them to obtain a comprehensive attention weight, which is multiplied with the enhanced image to obtain a weighted feature map;

[0021] The temporal attention module uses an LSTM network to process the feature graphs of continuous time series and outputs temporal attention features.

[0022] Preferably, the online learning operation is specifically based on a prototype network; the prototype network is divided into a training phase and a testing phase;

[0023] During the training phase, prototypes for each category are computed from the support set;

[0024] In the testing phase, the Euclidean distance between the new sample and each prototype is calculated, and the category corresponding to the prototype with the smallest distance is selected as the category of the new sample; the temporal attention feature is used as the sample feature.

[0025] Preferably, the hybrid posture control unit adopts a robust kinematics inverse solution based on Lie groups and an automatic switching algorithm of the force-position hybrid control mode to calculate the joint angles and the end effector posture of the robot, and the specific analysis steps are as follows:

[0026] The robust kinematics inverse solution based on Lie groups uses the Newton-Raphson iterative algorithm to solve the robot's joint angle vector according to the target object position and category information provided by the online meta-learning algorithm; if the joint angle vector is greater than its preset joint adjustment threshold, a posture control instruction is generated;

[0027] The automatic switching of the force-position mixed control mode adopts a fuzzy control algorithm, and adjusts the weights of force control and position control according to the force measured by the force sensor and the position error measured by the position sensor; sets the force measured by the force sensor and the position error measured by the position sensor. If any value of the force measured by the force sensor and the position error measured by the position sensor is greater than the error threshold, it indicates that there is a deviation between the actual motion position and the target position, and a force-position mixed control instruction is generated.

[0028] Preferably, the predictive maintenance unit monitors the state of the robot joints in real time, and adjusts the control parameters of the robot through the robot joint wear prediction model and the control parameter online optimization algorithm, and the specific steps are:

[0029] The robot arm joint wear prediction model uses an LSTM network to process the historical operation data of the robot arm joint and outputs a predicted value of the joint wear degree;

[0030] The control parameter online optimization algorithm adopts a method combining genetic algorithm and particle swarm algorithm, takes the robot arm motion performance and grasping accuracy as optimization goals, adjusts the robot's control parameters, and generates control parameter adjustment instructions when the control parameters are less than a preset allowable threshold.

[0031] Preferably, the robot positioning and grasping method based on the above system comprises the following steps:

[0032] The multimodal perception module acquires the object’s three-dimensional structure, material spectrum, environmental information, visual images, and tactile information;

[0033] The intelligent processing module performs image enhancement, dynamic attention modeling and online learning operations on the information output by the multimodal perception module;

[0034] The hybrid posture control unit calculates the robot's joint angles and end-effector posture according to the results of the intelligent processing module, using a robust kinematics inverse solution based on Lie groups and an automatic switching algorithm for force-position hybrid control modes, and generates corresponding control instructions.

[0035] The predictive maintenance unit monitors the status of the robot joints in real time and adjusts the robot's control parameters through the robot joint wear prediction model and control parameter online optimization algorithm;

[0036] The mechanical execution module receives the instructions issued by the adaptive control module and executes the positioning and grasping operation of the object.

[0037] Compared with the related art, the robot positioning and grasping system and method based on laser vision guidance provided by the present invention has the following beneficial effects:

[0038] 1. The present invention obtains rich environment and object information through a multimodal perception module, and combines the image enhancement, dynamic attention modeling and online learning operations of the intelligent processing module to improve the recognition accuracy of the target object and effectively reduce the influence of illumination and object surface characteristics on recognition.

[0039] 2. The hybrid posture control unit of the adaptive control module of the present invention adopts the robust kinematic inverse solution based on Lie group and the automatic switching algorithm of the force-position hybrid control mode. It can automatically switch the control mode according to the shape, material and other characteristics of the object to achieve stable grasping of different objects; the predictive maintenance unit effectively extends the service life of the robot arm and improves the grasping accuracy by real-time monitoring and optimization of control parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A principle block diagram of the robot positioning and grasping system provided by the present invention;

[0041] Figure 2A principle block diagram of the robot positioning and grasping system provided by the present invention;

[0042] Figure 3 This is a flowchart of the robot positioning and grasping method provided by the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms of "a", "an", "the" and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0045] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0046] Please refer to Figure 1-Figure 3 A robot positioning and grasping system based on laser vision guidance, including a multimodal perception module, an intelligent processing module, an adaptive control module and a mechanical execution module;

[0047] The multimodal perception module is used to obtain the object's three-dimensional structure, material spectrum, environmental information, visual images, and tactile information;

[0048] The intelligent processing module receives the information output by the multimodal perception module and performs intelligent processing operations; the intelligent processing operations include image enhancement operations, dynamic attention modeling operations and online learning operations;

[0049] The adaptive control module includes a hybrid posture control unit and a predictive maintenance unit;

[0050] The hybrid pose control unit uses a Lie group-based robust inverse kinematics and a force-position hybrid control mode automatic switching algorithm to calculate the joint angles and end effector poses of the robot;

[0051] The predictive maintenance unit monitors the joint states of the robotic arm in real time and adjusts the control parameters of the robot through a robotic arm joint wear prediction model and an online control parameter optimization algorithm;

[0052] The mechanical execution module receives instructions issued by the adaptive control module and performs object positioning and grasping operations, specifically:

[0053] When generating pose control instructions, it triggers the adjustment of the angles of each joint on the robot through drive motors and transmission mechanisms, so that the robotic arm moves to the target position along a predetermined trajectory;

[0054] When generating force-position hybrid control instructions, it triggers the fine-tuning of the drive motors and transmission mechanisms based on the force measured by the force sensor and the position error measured by the position sensor, so as to ensure that the robotic arm accurately reaches the target position. And when the robotic arm reaches the target position, the end effector adjusts the grasping force according to the force-position hybrid control instructions;

[0055] When generating control parameter adjustment instructions, it controls the adjustment of the corresponding control parameter p on the robot.

[0056] In this application, the multi-modal perception module includes a multi-spectral laser scanner, an environmental parameter sensor, a bionic compound eye camera, and a tactile feedback gripper;

[0057] The multi-spectral laser scanner adjusts the wavelength λ according to the material characteristics μ of the object to be recognized and the environmental light condition L through a wavelength dynamic adjustment module. The formula is λ = f(μ, λ), where f is a mapping function obtained through experimental data and machine learning model training, and is used to determine the optimal scanning wavelength under different material and light conditions; it scans the object to obtain the three-dimensional structure and material spectral information of the object to be recognized;

[0058] The environmental parameter sensor is used to monitor environmental parameters such as environmental temperature and humidity T and dust concentration C in real time;

[0059] The bionic compound eye camera uses a multi-channel fish-eye lens array for panoramic dynamic vision acquisition. The dynamic aperture adjustment unit automatically adjusts the aperture size A according to the environmental light intensity I through the formula A = g(I), where g is a non-linear function used to achieve the adaptation of the aperture size to the light intensity; the image preprocessing chip is used to perform preliminary preprocessing operations on the acquired images, including but not limited to image noise reduction and enhancement;

[0060] When the tactile feedback gripper contacts an object, the micro pressure sensor matrix measures the contact pressure P contact, the vibration tactile feedback module senses the vibration information V on the surface of the object vibration ; The contact pressure P with the object contact and the vibration information V of the object surface vibration Marked as tactile information.

[0061] In this application, the image enhancement operation specifically uses a generative adversarial network to optimize image quality, wherein the adversarial network is also called "GAN"; the adversarial network includes a generator G and a discriminator D;

[0062] The low-light multispectral image I collected by the multimodal perception module low Input the generator G of the generative adversarial network, perform multi-level feature extraction and reconstruction through the convolution layer C, pooling layer P and deconvolution layer Dc to generate an enhanced image The discriminator D is used to enhance the image and real high-quality image I read To make a judgment, the loss function of the generated adversarial network is:

[0063] Where E represents the expectation, and the parameters of the generator and the discriminator are adjusted iteratively until the loss function L GAN Convergence conditions; thus achieving better image enhancement effect;

[0064] It should be noted that using the existing public image database, which contains a large number of annotated and screened high-quality images, searching for high-quality images similar to objects in low-light multispectral images collected by the multimodal perception module in some databases specifically used for computer vision research (such as ImageNet), and selecting appropriate images from the public image database as true high-quality images through image retrieval algorithms according to the category, shape and other characteristics of the object, is already a prior art, and is not described in detail in this application; the convergence condition of the loss function is generally:

[0065] Observe the value of the loss function of the generative adversarial network. When the change of the loss function value is less than a preset threshold (such as 10) in multiple consecutive iterations (such as 100 iterations), -5 ), the loss function is considered to have converged; or the convergence can be judged by observing whether the performance of the generator and the discriminator is stable, which is also an existing mature technology and is not described in detail in this application.

[0066] In this application, the dynamic attention modeling operation specifically adopts a spatiotemporal attention mechanism; the spatiotemporal attention mechanism is composed of a spatial attention unit and a temporal attention unit working together;

[0067] Spatial attention module for image enhancement Process and calculate channel attention and spatial attention And combined to get the comprehensive attention weight W s , the formula indicates Where ⊙ represents element-by-element multiplication, and then the comprehensive attention weight W s With enhanced image Multiply to get the weighted feature map F s ;

[0068] The temporal attention module uses an LSTM network to process the feature map F of the continuous time series. s (t), output temporal attention feature F t , that is, F t =LSTM(F s (t)).

[0069] In this application, the online learning operation is specifically based on the prototype network; the prototype network is divided into a training phase and a testing phase;

[0070] In the training phase, the prototype representation P of each category is calculated from the support set S c , the formula indicates Where N c is the number of samples in category c, S c is a sample set of category c;

[0071] In the test phase, calculate the new sample x nex With each prototype P c The Euclidean distance d(x new ,P c ), the formula is in represents the jth eigenvalue in the new sample eigenvector, P c represents the prototype feature vector of category c, represents the jth eigenvalue in the prototype feature vector of category c, j represents the dimension index of the feature vector, from 1 to the total dimension number m of the feature vector; select the category corresponding to the prototype with the smallest distance as the category of the new sample; by using the temporal attention feature F t As sample features, it can realize fast and accurate recognition of new target objects.

[0072] It should be noted that the support set is obtained in the following way: when the robot arm successfully grasps a new target object, the relevant information of the object and its category information are added to the support set S, so that the support set is continuously enriched and improved, thereby improving the prototype network's ability to recognize new target objects.

[0073] In this application, the hybrid posture control unit uses a robust kinematics inverse solution based on Lie groups and an automatic switching algorithm for the force-position hybrid control mode to calculate the joint angles and end effector posture of the robot. The specific analysis steps are as follows:

[0074] The robust inverse kinematics solution based on Lie groups is based on the target object position T provided by the online meta-learning algorithm. d and category information, and solve the robot's joint angle vector θ through the Newton-Raphson iterative algorithm; the iterative formula represents θ k+1 =θ k -J + (T k -T d ), where θ k is the joint angle vector of the kth iteration, J + is the pseudo-inverse of the Jacobian matrix, T k is the posture of the robot end effector at the kth iteration; if the joint angle vector is greater than its preset joint adjustment threshold, a posture control instruction is generated;

[0075] The force-position hybrid control mode is automatically switched using a fuzzy control algorithm, based on the force F measured by the force sensor and the position error e measured by the position sensor. p , adjusting force control u f and position control u p The weight α is expressed by the formula α=fuzzy(F,e p ), where fuzzy represents the preset fuzzy rule base, which is specifically set according to experimental data and experience; the final control output u is u=αu f +(1-α)u p , thus achieving stable grasping of different objects;

[0076] Set the force measured by the force sensor and the position error threshold measured by the position sensor. If any value of the force measured by the force sensor and the position error measured by the position sensor is greater than the error threshold, it means that there is a deviation between the actual motion position and the target position, and a force-position mixed control instruction is generated.

[0077] In this application, the predictive maintenance unit monitors the state of the robot joints in real time, and adjusts the control parameters of the robot through the robot joint wear prediction model and the control parameter online optimization algorithm. The specific steps are as follows:

[0078] The robot joint wear prediction model uses the LSTM network to process the historical operation data X of the robot joint and output the predicted value of joint wear degree.

[0079] The control parameter online optimization algorithm adopts a method combining genetic algorithm and particle swarm algorithm, takes the robot's motion performance and grasping accuracy as the optimization target, adjusts the robot's control parameter p, and generates a control parameter adjustment instruction when the control parameter is less than the preset allowable threshold; specifically:

[0080] Obtain the grasping accuracy DS1, movement speed DS2, and movement stability DS3 of the robot arm;

[0081] The fitness function is f(p)=w1×DS1+w2×DS2+w3×DS3, where w1, w2, and w3 represent the weight coefficients corresponding to grasping accuracy, movement speed, and movement stability, respectively.

[0082] The control parameter p is set according to the actual situation of the robot. For example, when the control parameter is the motor drive current parameter, when the movement speed of the robot arm is lower than the set threshold, a control parameter adjustment instruction corresponding to the motor drive current parameter is generated and adjusted according to the control parameter p.

[0083] It should also be noted that the grasping accuracy DS1, movement speed DS2, and movement stability DS3 of the robot arm are conventional technologies, which are briefly described below in this application, and their acquisition methods are:

[0084] The grasping accuracy is obtained by visual detection method. High-precision visual sensors (such as industrial cameras) are arranged in the working area of ​​the robot arm. Before the robot arm grasps the target object, the initial position and posture information of the target object in the workspace are obtained, which is recorded as (x in ,y in ,z in ,m in ,n in ,j in ), (x in ,y in ,z in ) is the position coordinate, (m in ,n in ,j in ) is the posture angle; after the grasping is completed, the position and posture information of the object are obtained again (x fi ,y fi ,z fi ,m fi ,n fi ,j fi ), the grasping accuracy is measured by calculating the deviation between the actual grasping position and posture and the preset target position and posture. The position deviation calculation formula is Among them, x ta ,y ta 、z ta Respectively represent the coordinate values ​​of the preset target position in the x-axis, y-axis and z-axis directions; the attitude deviation d th It can be calculated by angle difference (such as using Euler angle or quaternion difference calculation); the grasping accuracy DS1 can be expressed as a ratio compared with the preset deviation threshold DS1n The formula indicates that when the actual deviation is smaller, DS1 is closer to 1, indicating higher grasping accuracy;

[0085] The motion speed DS2 can be obtained by the laser rangefinder sensor measurement method. Specifically, the laser rangefinder sensors are arranged around the robot arm workspace to measure the distance between the robot arm end effector and the sensor in real time. By analyzing the change of distance over time, the motion speed of the end effector is calculated. When the distances measured by the sensor at time t and t+Δt are d and d+Δd respectively, the speed The moving speed DS2 is obtained by taking the statistical average of the speed during the grasping process;

[0086] The motion stability DS3 can be obtained by vibration sensor measurement method, specifically:

[0087] Install a vibration sensor on the robot arm to collect the vibration signal of the robot arm during movement; perform spectrum analysis on the vibration signal to calculate the vibration amplitude FD and frequency fn; set the vibration amplitude threshold FDn. When the vibration amplitude is less than the threshold and the frequency is within the allowable range, use the formula The motion stability DS3 is calculated; when FD is closer to 0, the motion stability is closer to 1.

[0088] In the present application, the robot positioning and grasping method based on the above system includes the following steps:

[0089] The multimodal perception module acquires the object’s three-dimensional structure, material spectrum, environmental information, visual images, and tactile information;

[0090] The intelligent processing module performs image enhancement, dynamic attention modeling and online learning operations on the information output by the multimodal perception module;

[0091] The hybrid posture control unit calculates the robot's joint angles and end-effector posture according to the results of the intelligent processing module, using a robust kinematics inverse solution based on Lie groups and an automatic switching algorithm for force-position hybrid control modes, and generates corresponding control instructions.

[0092] The predictive maintenance unit monitors the status of the robot joints in real time and adjusts the robot's control parameters through the robot joint wear prediction model and control parameter online optimization algorithm;

[0093] The mechanical execution module receives the instructions issued by the adaptive control module and executes the positioning and grasping operation of the object.

[0094] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in this disclosure. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

[0095] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. The robot positioning and grasping system based on laser vision guidance is characterized by: It includes a multimodal perception module, an intelligent processing module, an adaptive control module and a mechanical execution module; The multimodal perception module is used to obtain the three-dimensional structure, material spectrum, environmental information, visual image and tactile information of the object; The intelligent processing module receives the information output by the multimodal perception module and performs intelligent processing operations; the intelligent processing operations include image enhancement operations, dynamic attention modeling operations and online learning operations; The adaptive control module includes a hybrid posture control unit and a predictive maintenance unit; The hybrid posture control unit uses a robust kinematics inverse solution based on Lie groups and an automatic switching algorithm of the force-position hybrid control mode to calculate the joint angles and the end effector posture of the robot and generate corresponding control instructions; The predictive maintenance unit monitors the state of the robot arm joints in real time, and adjusts the control parameters of the robot through the robot arm joint wear prediction model and the control parameter online optimization algorithm; The mechanical execution module receives the instruction from the adaptive control module and executes the positioning and grasping operation of the object; The instructions issued by the adaptive control module include posture control instructions, force-position mixed control instructions and control parameter adjustment instructions.

2. The robot positioning and grasping system based on laser vision guidance according to claim 1 is characterized in that: The multimodal perception module includes a multispectral laser scanner, an environmental parameter sensor, a bionic compound eye camera and a tactile feedback gripper; The multispectral laser scanner adjusts the wavelength through the wavelength dynamic adjustment module according to the material characteristics of the object to be identified and the ambient light conditions, scans the object, and obtains the three-dimensional structure and material spectrum information of the object; Environmental parameter sensors are used to monitor environmental temperature, humidity, and dust concentration in real time; The bionic compound eye camera uses a multi-channel fisheye lens array for panoramic dynamic visual acquisition, and the dynamic aperture adjustment unit automatically adjusts the aperture size according to changes in ambient light; the image preprocessing chip is used to perform preliminary preprocessing operations on the acquired images, including image noise reduction and enhancement; When the tactile feedback gripper contacts an object, a micro pressure sensor matrix and a vibration tactile feedback module provide tactile information.

3. The robot positioning and grasping system based on laser vision guidance according to claim 1 is characterized in that: The image enhancement operation specifically uses a generative adversarial network to optimize image quality; the adversarial network includes a generator and a discriminator; The low-light multispectral image collected by the multimodal perception module is input into the generator G of the generative adversarial network, and multi-level feature extraction and reconstruction are performed through the convolution layer C, pooling layer P and deconvolution layer Dc to generate an enhanced image; the discriminator D distinguishes the enhanced image from the real high-quality image, and adjusts the parameters of the generator and discriminator through continuous iteration until the image reaches the convergence condition.

4. The robot positioning and grasping system based on laser vision guidance according to claim 1 is characterized in that: The dynamic attention modeling operation specifically adopts a spatiotemporal attention mechanism; the spatiotemporal attention mechanism is composed of a spatial attention unit and a temporal attention unit working together; The spatial attention module processes the enhanced image, calculates the channel attention and the spatial attention, and combines them to obtain a comprehensive attention weight, which is multiplied with the enhanced image to obtain a weighted feature map; The temporal attention module uses an LSTM network to process the feature graphs of continuous time series and outputs temporal attention features.

5. The robot positioning and grasping system based on laser vision guidance according to claim 1 is characterized in that: The online learning operation is specifically based on a prototype network; the prototype network is divided into a training phase and a testing phase; During the training phase, prototypes for each category are computed from the support set; In the testing phase, the Euclidean distance between the new sample and each prototype is calculated, and the category corresponding to the prototype with the smallest distance is selected as the category of the new sample; the temporal attention feature is used as the sample feature.

6. The robot positioning and grasping system based on laser vision guidance according to claim 1 is characterized in that: The hybrid posture control unit uses a robust kinematics inverse solution based on Lie groups and an automatic switching algorithm for force-position hybrid control modes to calculate the joint angles and end effector posture of the robot. The specific analysis steps are as follows: The robust kinematics inverse solution based on Lie groups uses the Newton-Raphson iterative algorithm to solve the robot's joint angle vector according to the target object position and category information provided by the online meta-learning algorithm; if the joint angle vector is greater than its preset joint adjustment threshold, a posture control instruction is generated; The automatic switching of the force-position hybrid control mode adopts the fuzzy control algorithm, and adjusts the weights of force control and position control according to the force measured by the force sensor and the position error measured by the position sensor; Set the force measured by the force sensor and the position error threshold measured by the position sensor. If any value of the force measured by the force sensor and the position error measured by the position sensor is greater than the error threshold, it means that there is a deviation between the actual motion position and the target position, and a force-position mixed control instruction is generated.

7. The robot positioning and grasping system based on laser vision guidance according to claim 1 is characterized in that: The predictive maintenance unit monitors the state of the robot joints in real time, and adjusts the control parameters of the robot through the robot joint wear prediction model and the control parameter online optimization algorithm. The specific steps are as follows: The robot arm joint wear prediction model uses an LSTM network to process the historical operation data of the robot arm joint and outputs a predicted value of the joint wear degree; The control parameter online optimization algorithm adopts a method combining genetic algorithm and particle swarm algorithm, takes the robot arm motion performance and grasping accuracy as optimization goals, adjusts the robot's control parameters, and generates control parameter adjustment instructions when the control parameters are less than a preset allowable threshold.

8. A robot positioning and grasping method based on the system of any one of claims 1 to 7, comprising the following steps: The multimodal perception module acquires the object’s three-dimensional structure, material spectrum, environmental information, visual images, and tactile information; The intelligent processing module performs image enhancement, dynamic attention modeling and online learning operations on the information output by the multimodal perception module; The hybrid posture control unit calculates the robot's joint angles and end-effector posture according to the results of the intelligent processing module, using a robust kinematics inverse solution based on Lie groups and an automatic switching algorithm for force-position hybrid control modes, and generates corresponding control instructions. The predictive maintenance unit monitors the status of the robot joints in real time and adjusts the robot's control parameters through the robot joint wear prediction model and control parameter online optimization algorithm; The mechanical execution module receives the instructions issued by the adaptive control module and executes the positioning and grasping operation of the object.

Images