Supermarket robot self-adaptive flexible grabbing method and system based on visual guidance and reinforcement learning

Abstract

The invention relates to the technical field of robot grabbing, in particular to a supermarket robot self-adaptive flexible grabbing method and system based on visual guidance and reinforcement learning, and the system comprises a sensing and modeling module, a path planning and decision making module, a force control and feedback module, a self-adaptive adjustment module and a data storage and visualization module. Geometric and material characteristics of a target object are obtained through a binocular camera and a touch sensor, a motion track is optimized in combination with reinforcement learning, and precise force control is realized by using a six-dimensional force sensor. According to the method, the grabbing efficiency, the flexibility and the self-adaptive capacity of the robot in a complex dynamic scene can be improved, meanwhile, an intuitive visual monitoring means is provided, and the efficient and safe grabbing requirement is met.

Description

Technical Field

[0001] This invention belongs to the fields of robotics and artificial intelligence, specifically a method and system for adaptive compliant grasping of supermarket robots based on visual guidance and reinforcement learning. Background Technology

[0002] With the widespread application of robotics in logistics, retail, and other fields, supermarket robot grasping technology has gradually become a research hotspot. Existing robot grasping methods mainly rely on the cooperation of visual guidance and mechanical structures to achieve the recognition and grasping of target objects. However, in complex scenarios, such as densely packed and irregularly shaped goods on supermarket shelves, existing technologies often struggle to meet the requirements for efficient and compliant grasping.

[0003] A search revealed a robot grasping method with publication number CN107009391 B, published on May 12, 2023. This patent provides a robot grasping method based on a camera-moving mechanism. The method uses a camera to move along a preset trajectory to collect images of multiple preset positions on a sorting table, thereby achieving the identification and positioning of multiple products. This technical solution solves the problem of large space occupation in traditional multi-vision systems and improves grasping efficiency. However, this method is only suitable for static grasping tasks in fixed scenarios and lacks adaptability to dynamic environments. Furthermore, this method does not consider the compliance of the target object and potential collisions during the grasping process, which may lead to grasping failure or damage to the goods.

[0004] A search revealed a grasping method, grasping device, and robot with publication number CN113034526B, published on January 16, 2024. This patent proposes a grasping method based on environmental depth maps. It obtains a set of object contour points, extracts candidate grasping points, and combines reachability analysis and grasping quality assessment to determine the target grasping point to complete the grasping operation. This technical solution can grasp unmodeled objects placed in a disordered manner, improving grasping efficiency. However, this method mainly relies on static visual information and geometric analysis, lacking the ability to perceive the material properties of the target object (such as hardness), which may lead to improper control of the grasping force. Furthermore, this solution does not incorporate a reinforcement learning mechanism, making it difficult to achieve adaptive grasping optimization in complex environments, limiting its application in dynamic scenes.

[0005] The aforementioned problems indicate that existing robot grasping methods still have certain shortcomings in terms of dynamic environment adaptability, compliance control, and adaptive optimization. Therefore, this invention provides a visual guidance and reinforcement learning-based adaptive compliance grasping method and system for supermarket robots. The aim is to achieve efficient and compliant grasping of complex scenes in supermarket environments by combining visual guidance, tactile feedback, and reinforcement learning algorithms, thereby improving the robot's adaptability and grasping success rate in dynamic environments and meeting the retail sector's demand for intelligent grasping technology. Summary of the Invention

[0006] In view of the aforementioned existing problems, the present invention is proposed.

[0007] Therefore, this invention provides an adaptive compliant grasping method and system for supermarket robots based on visual guidance and reinforcement learning, addressing the shortcomings of existing robot grasping technologies in terms of dynamic environment adaptability, compliant control, and adaptive optimization. Specifically, existing technologies struggle to meet the demands for efficient and compliant grasping in complex scenarios, have limited perception capabilities regarding the material properties of target objects, and lack real-time adjustment capabilities in dynamic environments.

[0008] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0009] In a first aspect, the present invention provides an adaptive compliant grasping method and system for supermarket robots based on visual guidance and reinforcement learning, which includes the following modules:

[0010] The perception and modeling module acquires depth and color images of the target object using a binocular camera, and generates a three-dimensional geometric model of the target object using a point cloud processing algorithm. The three-dimensional geometric model of the target object is used to describe the shape features and spatial distribution characteristics of the object. Furthermore, the module acquires surface stress distribution data of the target object using a tactile sensor array, and calculates the material hardness coefficient of the target object using an elasticity model. The material hardness coefficient is used to guide the adjustment of the subsequent gripping force.

[0011] The path planning and decision-making module, based on the three-dimensional geometric model of the target object, uses the Rapid Expanding Random Tree (RRT) algorithm to generate the initial motion trajectory of the robotic arm. The initial motion trajectory contains multiple key nodes, each corresponding to the target position and orientation of the robotic arm's end effector. Furthermore, combining the material hardness coefficient of the target object, a reinforcement learning algorithm is introduced to optimize the initial motion trajectory. The reinforcement learning algorithm uses the grasping success rate and the collision probability during the grasping process as reward functions, and generates the optimal motion trajectory through iterative training.

[0012] The force control and feedback module monitors the contact force between the robotic arm's end effector and the target object in real time using a six-dimensional force sensor, and decomposes the contact force into normal force and tangential force. The normal force is used to assess gripping stability, and the tangential force is used to detect the presence of slippage risk. Furthermore, based on the material hardness coefficient and real-time contact force data, a proportional-integral-derivative (PID) controller is used to dynamically adjust the gripping force of the end effector. The adjustment range of the gripping force is limited by a preset safety threshold to avoid damage to the target object due to excessive force.

[0013] The adaptive adjustment module continuously collects real-time position change data of the target object through a binocular camera during the robotic arm's grasping task, and analyzes the motion trend of the target object in combination with an optical flow algorithm. The motion trend is used to predict the displacement and rotation angle of the target object in the future. Furthermore, the robot arm's motion trajectory is corrected online based on the prediction results to ensure that the grasping action can adapt to the dynamic changes of the target object.

[0014] The data storage and visualization module stores key data from the grasping task through an embedded database, including the target object's 3D geometric model, material hardness coefficient, initial motion trajectory, optimal motion trajectory, and force feedback data during the grasping process. The key data is organized using timestamps as indexes for easy retrospective analysis. Furthermore, the module displays the real-time status of the grasping task through a graphical user interface, including the current position of the robotic arm, the predicted motion trend of the target object, and the change curve of the gripping force.

[0015] The specific implementation methods of each module in this invention are as follows:

[0016] The perception and modeling module works in collaboration with a binocular camera and a tactile sensor array. The binocular camera is mounted on a fixed bracket above the robotic arm base, while the tactile sensor array is integrated into the gripping surface of the robotic arm's end effector. The binocular camera and the tactile sensor array transmit data synchronously through a high-speed communication interface, ensuring that the acquired image data and stress distribution data are consistent in time.

[0017] The path planning and decision-making module adopts a distributed computing architecture, in which the Rapidly Expanding Random Tree (RRT) algorithm runs on a high-performance computing unit, while the reinforcement learning algorithm is deployed in a dedicated neural network accelerator; the two interact with each other through shared memory to ensure that the initial motion trajectory and the optimized motion trajectory can be seamlessly connected.

[0018] The six-dimensional force sensor in the force control and feedback module is fixed to the base of the end effector of the robotic arm by bolts. Its signal output terminal is connected to the input terminal of the PID controller through a shielded cable. The output terminal of the PID controller is connected to the motor of the end effector through the drive circuit, forming a closed-loop control system.

[0019] The binocular camera and optical flow algorithm in the adaptive adjustment module run on the same embedded processor, which is connected to the main control unit of the robotic arm through a serial communication interface. The main control unit adjusts the motion commands in real time according to the received prediction results to ensure that the robotic arm can quickly respond to the dynamic changes of the target object.

[0020] The embedded database in the data storage and visualization module is deployed on a solid-state drive inside the robotic arm control box, while the graphical user interface runs on a separate touchscreen terminal. The touchscreen terminal is connected to the robotic arm control box via a wireless local area network to enable remote access to and visualization of the data.

[0021] This invention, through the aforementioned technical solution, solves the problems of low grasping efficiency, poor compliance, and insufficient dynamic adaptability in existing technologies under complex scenarios. Specifically, the perception and modeling module combines visual and tactile information to provide a comprehensive description of the geometric and physical characteristics of the target object; the path planning and decision-making module achieves adaptive optimization of the motion trajectory through reinforcement learning algorithms; the force control and feedback module achieves precise adjustment of the gripping force through a closed-loop control system; the adaptive adjustment module ensures that the grasping action maintains high precision in dynamic environments by monitoring and predicting the motion trend of the target object in real time; and the data storage and visualization module provides an intuitive means of displaying the system's operating status, facilitating monitoring and maintenance by operators. Attached Figure Description

[0022] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0023] Figure 1 This is a schematic diagram of the overall system structure of the present invention, showing the connection relationship between the perception and modeling module, the path planning and decision-making module, the force control and feedback module, the adaptive adjustment module, and the data storage and visualization module.

[0024] Figure 2 The flowchart for the perception and modeling module shows the process of the binocular camera acquiring images of the target object and generating a 3D geometric model, as well as the steps of the tactile sensor array acquiring stress distribution data and calculating the material hardness coefficient.

[0025] Figure 3 This is an optimization flowchart for the path planning and decision-making module, including the logical framework for initial trajectory generation, trajectory optimization using reinforcement learning algorithms, and reward function evaluation.

[0026] Figure 4 This is a closed-loop control diagram of the force control and feedback module, illustrating how a six-dimensional force sensor monitors the contact force and dynamically adjusts the clamping force through a PID controller.

[0027] Figure 5 This is a functional structure diagram of the data storage and visualization module, showing the components of the embedded database storing key data and the graphical user interface displaying the capture status in real time.

[0028] The attached figures are labeled as follows:

[0029] 1. Binocular camera; 2. Tactile sensor array; 3. Six-dimensional force sensor; 4. PID controller; 5. Robotic arm end effector; 6. Embedded database; 7. Graphical user interface. Detailed Implementation

[0030] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] This invention provides an adaptive compliant grasping method and system for supermarket robots based on visual guidance and reinforcement learning. It achieves efficient and compliant grasping of target objects through the collaborative work of multiple modules. The specific implementation of this invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0032] like Figure 1 As shown, the overall structure of this system includes a perception and modeling module, a path planning and decision-making module, a force control and feedback module, an adaptive adjustment module, and a data storage and visualization module. These modules work together through hardware and software to complete the grasping task. Specifically, the binocular camera 1 is mounted on a fixed bracket above the robotic arm base; the tactile sensor array 2 is integrated into the gripping surface of the robotic arm end effector 5; the six-dimensional force sensor 3 is bolted to the base of the robotic arm end effector 5; the PID controller 4 is connected to the signal output terminal of the six-dimensional force sensor 3 via a shielded cable; the embedded database 6 is deployed on a solid-state drive inside the robotic arm control box; and the graphical user interface 7 runs on a separate touchscreen terminal and is connected to the robotic arm control box via a wireless local area network.

[0033] The workflow of the perception and modeling module is as follows: Figure 2As shown, a binocular camera 1 acquires depth and color images of the target object. The depth image is used to generate a 3D geometric model of the target object, and the color image is used to assist in identifying the color features of the target object. The binocular camera 1 transmits the acquired image data to an embedded processor, which generates a 3D geometric model of the target object using a point cloud processing algorithm. This model describes the shape features and spatial distribution characteristics of the target object, providing basic information for subsequent path planning. Simultaneously, a tactile sensor array 2 acquires stress distribution data on the surface of the target object. This data is transmitted synchronously to the embedded processor via a high-speed communication interface, along with the image data acquired by the binocular camera 1. The embedded processor calculates the material hardness coefficient of the target object using an elasticity model. This coefficient guides the adjustment of the subsequent gripping force. The data synchronization between the tactile sensor array 2 and the binocular camera 1 ensures temporal consistency between the acquired image data and the stress distribution data, thereby improving the accuracy of the material hardness coefficient calculation.

[0034] The optimization process of the path planning and decision-making module is as follows: Figure 3 As shown, based on the 3D geometric model of the target object, the Rapid Expanding Random Tree (RRT) algorithm runs on a high-performance computing unit to generate the initial motion trajectory of the robotic arm. The initial motion trajectory contains multiple key nodes, each corresponding to the target position and orientation of the robotic arm's end effector 5. A reinforcement learning algorithm is deployed in a dedicated neural network accelerator, interacting with the high-performance computing unit via shared memory to ensure seamless integration between the initial and optimized motion trajectories. The reinforcement learning algorithm uses the success rate of grasping and the probability of collisions during the grasping process as reward functions, generating the optimal motion trajectory through iterative training. During training, the reinforcement learning algorithm continuously adjusts the weights of the reward function based on data from historical grasping tasks, making the generated motion trajectory more consistent with actual needs. The final generated optimal motion trajectory is transmitted to the robotic arm's main control unit to guide the robotic arm's grasping actions.

[0035] The closed-loop control diagram of the force control and feedback module is shown below. Figure 4As shown, the six-dimensional force sensor 3 monitors the contact force between the robotic arm's end effector 5 and the target object in real time, decomposing the contact force into normal and tangential forces. The normal force is used to assess gripping stability, while the tangential force is used to detect the risk of slippage. The signal output of the six-dimensional force sensor 3 is connected to the input of the PID controller 4 via a shielded cable. The PID controller 4 dynamically adjusts the gripping force of the end effector 5 based on the material hardness coefficient and real-time contact force data. The adjustment range of the gripping force is limited by a preset safety threshold to prevent damage to the target object due to excessive force. The output of the PID controller 4 is connected to the motor of the end effector 5 via a drive circuit, forming a closed-loop control system. During the gripping process, the six-dimensional force sensor 3 continuously monitors changes in the contact force and transmits the data to the PID controller 4. The PID controller 4 adjusts the gripping force in real time based on the received data to ensure the smoothness of the gripping action.

[0036] The adaptive adjustment module works as follows: During the robotic arm's grasping task, the binocular camera 1 continuously collects real-time position change data of the target object and transmits the data to the embedded processor. The embedded processor, combined with an optical flow algorithm, analyzes the motion trend of the target object and predicts its displacement and rotation angle over a future period. The prediction results are transmitted to the robotic arm's main control unit via a serial communication interface. The main control unit adjusts the motion commands in real time based on the received prediction results, ensuring that the robotic arm can quickly respond to the dynamic changes of the target object. The binocular camera 1 and the optical flow algorithm run on the same embedded processor, which is connected to the robotic arm's main control unit via a serial communication interface, ensuring that the prediction results are transmitted to the main control unit in a timely manner.

[0037] The functional structure of the data storage and visualization module is as follows: Figure 5 As shown, the embedded database 6 stores key data from the grasping task, including the target object's 3D geometric model, material hardness coefficient, initial motion trajectory, optimal motion trajectory, and force feedback data during the grasping process. This key data is organized using timestamps as indexes for easy retrospective analysis. The graphical user interface 7 runs on a separate touchscreen terminal, connected to the robotic arm control box via a wireless LAN, enabling remote access and visualization of the data. The graphical user interface 7 displays the real-time status of the grasping task, including the robotic arm's current position, the predicted motion trend of the target object, and the change curve of the gripping force. Operators can monitor the execution of the grasping task through the graphical user interface 7 and manually intervene when necessary.

[0038] In a specific application scenario, suppose a supermarket robot needs to grasp a glass bottle placed on a shelf. First, a binocular camera 1 acquires depth and color images of the glass bottle, generating a 3D geometric model of the bottle. A tactile sensor array 2 collects stress distribution data on the surface of the glass bottle and calculates the material hardness coefficient of the bottle using an elasticity model. The path planning and decision-making module generates the initial motion trajectory of the robotic arm based on the 3D geometric model of the glass bottle and optimizes the trajectory using a reinforcement learning algorithm. The six-dimensional force sensor 3 in the force control and feedback module monitors the contact force between the robotic arm's end effector 5 and the glass bottle in real time, and the PID controller 4 dynamically adjusts the gripping force based on the material hardness coefficient and real-time contact force data. During the grasping process, the binocular camera 1 continuously monitors the real-time position changes of the glass bottle, and the embedded processor, combined with an optical flow algorithm, predicts the movement trend of the glass bottle and transmits the prediction results to the robotic arm's main control unit. The main control unit adjusts the motion commands in real time based on the prediction results. An embedded database 6 stores key data in the grasping task, and a graphical user interface 7 displays the real-time status of the grasping task, facilitating monitoring and maintenance by operators.

[0039] The above embodiments describe in detail the technical solution and specific implementation steps of the present invention. Through the collaborative work of various modules, the problems of low grasping efficiency, poor compliance, and insufficient dynamic adaptability in complex scenarios of the prior art are solved. The connection relationships, positional relationships, and mutual cooperation relationships between the modules are fully disclosed to ensure that those skilled in the art can implement the technology according to the contents of the specification.

[0040] To enable those skilled in the art to fully understand and implement this invention, the specific implementation principle of this invention will be further explained below in conjunction with a specific application scenario.

[0041] In a supermarket environment, suppose a robot needs to grasp a glass bottle placed on a shelf. First, a binocular camera 1 is mounted on a fixed bracket above the robotic arm base to acquire depth and color images of the target glass bottle. After receiving this image data, the embedded processor generates a 3D geometric model of the glass bottle using a point cloud processing algorithm. This model describes the shape features of the glass bottle and its spatial distribution characteristics, providing basic information for subsequent path planning. Simultaneously, a tactile sensor array 2 is integrated into the gripping surface of the robotic arm's end effector 5, acquiring real-time stress distribution data on the glass bottle surface and transmitting this data to the embedded processor via a high-speed communication interface. The embedded processor, combined with an elasticity model, calculates the hardness coefficient of the glass bottle's material. This coefficient guides the adjustment of the gripping force during subsequent grasping. The synchronous design between the tactile sensor array 2 and the binocular camera 1 ensures the temporal consistency of image data and stress distribution data, thereby improving the accuracy of the hardness coefficient calculation.

[0042] Next, the path planning and decision-making module, based on the generated 3D geometric model of the glass bottle, uses the Rapid Expanding Random Tree (RRT) algorithm, running on a high-performance computing unit, to generate the initial motion trajectory of the robotic arm. The initial motion trajectory contains multiple key nodes, each corresponding to the target position and orientation of the robotic arm's end effector 5. A reinforcement learning algorithm, deployed in a dedicated neural network accelerator, interacts with the high-performance computing unit via shared memory, ensuring seamless integration between the initial and optimized motion trajectories. The reinforcement learning algorithm uses the success rate of grasping and the probability of collisions during the grasping process as reward functions, generating the optimal motion trajectory through iterative training. During training, the reinforcement learning algorithm continuously adjusts the weights of the reward function based on data from historical grasping tasks, making the generated motion trajectory more consistent with actual needs. Finally, the generated optimal motion trajectory is transmitted to the robotic arm's main control unit to guide the robotic arm's grasping actions.

[0043] During the gripping process, a six-dimensional force sensor 3 is bolted to the base of the robotic arm's end effector 5, monitoring the contact force between the end effector 5 and the glass bottle in real time. The six-dimensional force sensor 3 decomposes the contact force into normal and tangential forces. The normal force is used to assess gripping stability, while the tangential force is used to detect any risk of slippage. A PID controller 4 is connected to the signal output of the six-dimensional force sensor 3 via a shielded cable, dynamically adjusting the clamping force of the end effector 5 based on the material hardness coefficient and real-time contact force data. The adjustment range of the clamping force is limited by a preset safety threshold to prevent damage to the glass bottle due to excessive force. The output of the PID controller 4 is connected to the motor of the end effector 5 via a drive circuit, forming a closed-loop control system. During the gripping process, the six-dimensional force sensor 3 continuously monitors changes in the contact force and transmits the data to the PID controller 4. The PID controller 4 adjusts the clamping force in real time based on the received data, thereby ensuring the smoothness of the gripping action.

[0044] Meanwhile, the adaptive adjustment module continuously collects real-time position change data of the glass bottle through the binocular camera 1 and transmits this data to the embedded processor. The embedded processor, combined with an optical flow algorithm, analyzes the movement trend of the glass bottle and predicts its displacement and rotation angle over a future period. The prediction results are transmitted to the robotic arm's main control unit via a serial communication interface. The main control unit adjusts the motion commands in real time based on the received prediction results, ensuring that the robotic arm can quickly respond to the dynamic changes of the glass bottle. The binocular camera 1 and the optical flow algorithm run on the same embedded processor, which is connected to the robotic arm's main control unit via a serial communication interface, ensuring that the prediction results are transmitted to the main control unit in a timely manner.

[0045] Throughout the grasping task, the embedded database 6 stores key data, including the 3D geometric model of the glass bottle, its material hardness coefficient, initial trajectory, optimal trajectory, and force feedback data during the grasping process. This data is organized using timestamps for easy retrospective analysis. The graphical user interface 7 runs on a separate touchscreen terminal, connected to the robotic arm control box via a wireless LAN, enabling remote access and visualization of the data. The graphical user interface 7 displays the real-time status of the grasping task, including the current position of the robotic arm, the predicted movement trend of the glass bottle, and the curve of the gripping force variation. Operators can monitor the execution of the grasping task through the graphical user interface 7 and intervene manually when necessary.

[0046] Through the above steps, the perception and modeling module provides a comprehensive description of the target object's geometric and physical characteristics; the path planning and decision-making module achieves adaptive optimization of the motion trajectory; the force control and feedback module ensures precise adjustment of the gripping force; and the adaptive adjustment module guarantees high precision in the grasping action through real-time monitoring and prediction of the target object's motion trend. The data storage and visualization module provides an intuitive display of the system's operating status, facilitating monitoring and maintenance by operators. The collaborative work of these modules solves the problems of low grasping efficiency, poor compliance, and insufficient dynamic adaptability in complex scenarios of existing technologies, ensuring that those skilled in the art can implement this technology according to the instructions.

[0047] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method and system for adaptive compliant grasping of a supermarket robot based on visual guidance and reinforcement learning, characterized in that: The system comprises the following modules: a perception and modeling module that acquires depth images and color images of a target object through a binocular camera (1), generates a three-dimensional geometric model of the target object by combining point cloud processing algorithms, and acquires surface stress distribution data of the target object through a tactile sensor array (2) and calculates a material hardness coefficient; a path planning and decision-making module that generates an initial motion trajectory of a robot arm based on the three-dimensional geometric model of the target object using a rapidly-exploring random tree algorithm, and optimizes the initial motion trajectory through a reinforcement learning algorithm; a force control and feedback module that monitors contact force between an end effector (5) of the robot arm and the target object through a six-axis force sensor (3), decomposes the contact force into normal force and tangential force, and dynamically adjusts the clamping force using a proportional-integral-derivative controller; an adaptive adjustment module that acquires real-time position change data of the target object through the binocular camera (1), predicts the displacement and rotation angle of the target object by combining an optical flow algorithm, and corrects the motion trajectory of the robot arm online; a data storage and visualization module that stores key data in a grab task through an embedded database (6) and displays the real-time state of the grab task through a graphical user interface (7).

2. The supermarket robot adaptive compliant grasping method and system based on visual guidance and reinforcement learning of claim 1, wherein: In the perception and modeling module, the binocular camera (1) is installed on a fixed support above the base of the robot arm, the tactile sensor array (2) is integrated on the clamping surface of the end effector (5) of the robot arm, and the binocular camera (1) and the tactile sensor array (2) realize synchronous data transmission through a high-speed communication interface.

3. The supermarket robot adaptive compliant grasping method and system based on visual guidance and reinforcement learning of claim 1, wherein: In the path planning and decision-making module, the rapidly-exploring random tree algorithm runs on a high-performance computing unit, the reinforcement learning algorithm is deployed in a dedicated neural network accelerator, and the two interact through shared memory.

4. The vision-guided and reinforcement learning based supermarket robot adaptive compliant grasping method and system of claim 1, wherein: In the force control and feedback module, the six-axis force sensor (3) is fixed on the base of the end effector (5) of the robot arm through bolts, the signal output end is connected to the input end of the proportional-integral-derivative controller through a shielded cable, and the output end of the proportional-integral-derivative controller is connected to the motor of the end effector (5) through a drive circuit.

5. The vision-guided and reinforcement learning based supermarket robot adaptive compliant grasping method and system of claim 1, wherein: In the adaptive adjustment module, the binocular camera (1) and the optical flow algorithm run on the same embedded processor, and the embedded processor is connected to the main control unit of the robot arm through a serial communication interface.

6. The vision-guided and reinforcement learning based supermarket robot adaptive compliant grasping method and system of claim 1, wherein: In the data storage and visualization module, the embedded database (6) is deployed on a solid state disk in the robot arm control box, and the graphical user interface (7) runs on an independent touch screen terminal and is connected to the robot arm control box through a wireless local area network.

7. The vision-guided and reinforcement learning based supermarket robot adaptive compliant grasping method and system of claim 1, wherein: In the perception and modeling module, the material hardness coefficient is calculated through an elastic mechanics model to guide the adjustment of the clamping force.

8. The vision-guided and reinforcement learning based supermarket robot adaptive compliant grasping method and system of claim 1, wherein: In the path planning and decision-making module, the reinforcement learning algorithm takes the success rate of grabbing and the collision probability in the grabbing process as the reward function, and generates the optimal motion trajectory through iterative training.

9. The vision-guided and reinforcement learning based supermarket robot adaptive compliant grasping method and system of claim 1, wherein: In the force control and feedback module, the adjustment range of the clamping force is limited by a preset safety threshold, and the safety threshold is determined according to the material hardness coefficient and real-time contact force data.

10. The vision-guided and reinforcement learning based supermarket robot adaptive compliant grasping method and system of claim 1, wherein: The data storage and visualization module indexes and organizes key data, including a three-dimensional geometric model of a target object, a material hardness coefficient, an initial motion trajectory, an optimal motion trajectory and force feedback data during a grabbing process, by time stamp.

Images