A visual guidance system for mobile grasping devices

Abstract

The present invention provides a visual guidance system for a mobile grasping device, comprising a data acquisition and preprocessing module, the data acquisition and preprocessing module being used to acquire image data and preprocess the acquired image data, and transmitting the preprocessed image data to an instance segmentation module, the instance segmentation module being used to identify and segment a target object in the preprocessed image data, and outputting the pixel coordinate position and mask information of the target object to a three-dimensional positioning module, the three-dimensional positioning module being used to calculate the spatial position of the target object, and transmitting the spatial position information of the target object to an execution module, which accurately guides the grasping of the target object through the spatial position information of the target object. The visual guidance system for a mobile grasping device in the present invention can more accurately perform spatial positioning of the target object, and due to the use of technologies such as deep learning, instance segmentation, and spatial point cloud, the time required for the visual guidance system to adapt to the mobile grasping device is shorter.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a vision guidance system for a mobile grasping device. Background Art

[0002] With the rapid development of the manufacturing industry, in order to increase the speed of production and processing, production and processing equipment is becoming more and more intelligent and automated.

[0003] Existing technologies for visual guidance systems for mobile grippers require specialized engineering teams to develop customized solutions based on specific application scenarios. This approach requires significant time to adapt to different mobile grippers, incurring significant time and costs during large-scale factory automation upgrades, impacting normal factory production. Furthermore, in recent years, researchers have focused on lightweighting target recognition algorithms and constraining algorithms for calculating target poses. This has resulted in mobile grippers requiring specific placement requirements for loading objects, and this approach also provides low accuracy in determining the spatial pose of the target. Summary of the Invention

[0004] Based on this, the purpose of the present invention is to provide a visual guidance system for a mobile grasping device, which is used to solve the problems in the prior art that the visual guidance system requires a lot of time to adapt to the mobile grasping device, needs to constrain the placement of the target object, and the spatial posture solution accuracy of the grasped target object is not high.

[0005] The present invention proposes a visual guidance system for a mobile grasping device, comprising a data acquisition and preprocessing module, an instance segmentation module, a three-dimensional positioning module, and an execution module;

[0006] The data acquisition and preprocessing module is used to acquire image data, preprocess the acquired image data, and transmit the preprocessed image data to the instance segmentation module;

[0007] The instance segmentation module is used to identify and segment the target object in the preprocessed image data, and output the pixel coordinate position and mask information of the target object to the three-dimensional positioning module;

[0008] The three-dimensional positioning module is used to receive the pixel coordinate position and mask information of the target object, calculate the spatial position of the target object, and transmit the spatial position information of the target object to the execution module;

[0009] The execution module is used to receive the spatial posture information of the target object and grasp the target object according to the spatial posture information of the target object.

[0010] Compared with the prior art, the beneficial effect of the present invention is that the image acquisition and preprocessing module can acquire image data of the target area and preprocess it, eliminating the noise data of the image, so that the instance segmentation module can accurately identify and segment the pixel coordinate position and mask information of the target object. The three-dimensional positioning module can calculate the spatial position of the target object through the pixel coordinate position and mask information, so that it can accurately locate the target object in space, thereby providing the execution module with the position of the target object in space, so that the execution module can accurately grasp the target object. Because the visual guidance system adopts technologies such as instance segmentation, the guidance system can quickly adapt to the grasping device and guide the grasping device to accurately grasp the target object.

[0011] Furthermore, the execution module includes a logic controller and an execution unit, and the logic controller is used to receive the spatial posture information of the target object and control and guide the execution unit to perform a grasping operation.

[0012] Furthermore, the data acquisition and preprocessing module includes an image acquisition unit and a data preprocessing unit, the image acquisition unit is used to acquire the image data and transmit the acquired image data to the data preprocessing unit, the image data at least includes an RGB image and a depth image;

[0013] The data preprocessing unit is used to preprocess the RGB image and the depth image, and transmit the preprocessed RGB image to the instance segmentation module, and transmit the preprocessed depth map to the three-dimensional positioning module.

[0014] Furthermore, the instance segmentation module includes a feature extraction unit, a RolAlign layer, an RPN network unit and a segmentation unit;

[0015] The feature extraction unit is used to extract feature information from the RGB image;

[0016] The RolAlign layer is used to align the feature information with the RGB image;

[0017] The RPN network unit is used to identify the feature information corresponding to the target object in the feature information, and transmit the feature information corresponding to the target object to the RolAlign layer;

[0018] The segmentation unit is used to segment the mask information of the target object in the feature information.

[0019] Furthermore, the instance segmentation module also includes a fully connected layer and a target detection unit. The fully connected layer is used to convolutionally sample the aligned feature information to generate a corresponding feature vector and output it to the target detection unit. The target detection unit calculates the pixel coordinate position of the target object based on the feature vector and outputs the pixel coordinate position to the three-dimensional positioning module.

[0020] Furthermore, the instance segmentation module also includes a classification unit, which is used to calculate the classification result of the target object based on the feature vector.

[0021] Furthermore, the three-dimensional positioning module includes a point cloud segmentation unit, a point cloud registration unit and a template point cloud unit;

[0022] The point cloud segmentation unit extracts the spatial point cloud of the target object in the depth map according to the mask information of the target object. The point cloud registration unit matches the spatial point cloud with the multiple template point clouds transmitted by the template point cloud unit, and selects the template point cloud with the highest similarity among the multiple template point clouds for ICP calculation, and performs matrix multiplication on the calculated value and the template point cloud with the highest similarity to obtain the spatial pose of the target object.

[0023] Furthermore, the visual guidance system further includes a posture optimization module, which includes a target tracking unit and a posture graph optimization unit;

[0024] The target tracking unit is used to obtain all spatial poses of the target object transmitted by the three-dimensional positioning module before the execution unit grasps the target object, and calculate the spatial pose information of the target object based on all spatial poses of the target object;

[0025] The pose graph optimization unit is used to optimize the spatial pose information of the target object to obtain the final grasping pose of the execution unit.

[0026] Furthermore, the vision guidance system further includes a calibration module, and the calibration module is used to calibrate the execution unit and the vision guidance system.

[0027] Furthermore, the visual guidance system further includes a trigger module, which is used to trigger the image acquisition and preprocessing module to enable the image acquisition and preprocessing module to acquire the image data and preprocess the image data. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a workflow diagram of the visual guidance system of the present invention;

[0029] Figure 2is a workflow diagram of the instance segmentation module of the present invention;

[0030] Figure 3 This is a workflow diagram of the three-dimensional positioning module of the present invention;

[0031] Description of main component symbols:

[0032]

[0033]

[0034] The following specific embodiments will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0035] To facilitate understanding of the present invention, the present invention will be described more fully below with reference to the accompanying drawings. The drawings illustrate several embodiments of the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the present invention.

[0036] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be an intermediate element. When an element is referred to as being "connected to" another element, it may be directly connected to the other element or there may be an intermediate element. The terms "vertical," "horizontal," "left," "right," and similar expressions used herein are for illustrative purposes only.

[0037] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this invention pertains. The terms used in this specification of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0038] See also Figure 1 , shown is a visual guidance system for a mobile grasping device in an embodiment of the present invention, including a data acquisition and preprocessing module 30, an instance segmentation module 40, a three-dimensional positioning module 50 and an execution module 60.

[0039] The data acquisition and preprocessing module 30 is used to acquire image data, preprocess the acquired image data, and transmit the preprocessed image data to the instance segmentation module 40;

[0040] The instance segmentation module 40 is used to identify and segment the target object in the pre-processed image data, and output the pixel coordinate position 408 and mask information 405 of the target object to the 3D positioning module 50;

[0041] The three-dimensional positioning module 50 is used to receive the pixel coordinate position 408 and the mask information 405 of the target object, calculate the spatial position 503 of the target object, and transmit the spatial position 503 information of the target object to the execution module 60;

[0042] The execution module 60 is used to receive the spatial position 503 information of the target object and grasp the target object according to the spatial position 503 information of the target object.

[0043] The data acquisition and preprocessing module 30 acquires an image of the target area to be captured and preprocesses the image data information. Preprocessing the image data information can eliminate some noise data and improve the accuracy of the depth map data. The processed image data is then transmitted to the instance segmentation module 40. After the processed image data is transmitted from the data acquisition and preprocessing module 30 to the instance segmentation module 40, the instance segmentation module 40 identifies and segments the pixel coordinate position 408 and mask information 405 of the target in the image data through deep learning network inference, and transmits the pixel coordinate position 408 and mask information 405 of the target to the three-dimensional positioning module 50. After receiving the pixel coordinate position 408 and mask information 405 of the target, the three-dimensional positioning module 50 performs calculations within the three-dimensional positioning module 50, so that the three-dimensional positioning module 50 can accurately calculate the spatial posture 503 of the target. After the information of the spatial posture 503 of the target is calculated, it is transmitted to the execution module 60. After the execution module 60 receives the spatial position 503 information of the target object transmitted by the three-dimensional positioning module 50, the execution module 60 can accurately grasp the target object according to the guidance of the spatial position 503 information of the target object. In addition, the visual guidance system can directly process and collect image data of the target area, process the image data, and then calculate the spatial position 503 of the target object. The grasping device can accurately grasp the target object based on the spatial position 503 of the target object. This method of processing the target image data allows the visual guidance system to be used directly without adapting to the grasping device, thereby saving the time of adapting the mobile grasping device to the visual guidance system, and thus effectively saving time and cost.

[0044] See also Figure 1In this embodiment, the execution module 60 includes a logic controller 601 and an execution unit 602. The logic controller 601 can also control the execution unit 602 to perform a grasping operation. The execution unit 602 can be regarded as the grasping end of the grasping device. It can be understood that the logic controller 601 can control the grasping end of the grasping device to grasp the target object.

[0045] See also Figure 2 In this embodiment, the data acquisition and preprocessing module 30 includes an image acquisition unit and a data preprocessing unit. The image acquisition unit is used to acquire the image data and transmit the acquired image data to the data preprocessing unit. The image data includes an RGB image and a depth map. The data preprocessing unit is used to preprocess the RGB image and the depth map, and transmit the preprocessed RGB image to the instance segmentation module 40, and the preprocessed depth map to the three-dimensional positioning module. The image acquisition unit acquires the image of the area of the target object to be captured, and then transmits the acquired image data to the data preprocessing unit. The data preprocessing unit preprocesses the acquired image data of the area of the target object. The preprocessing can eliminate some noise data, improve the accuracy of the depth map data, and enable the instance segmentation module 40 to process the image data more accurately to avoid unnecessary errors. The depth map is a depth value matrix corresponding to the image pixels, and the RGB map is color image data.

[0046] See also Figure 2 In this embodiment, the instance segmentation module 40 includes a feature extraction unit 401, a RolAlign layer 403, an RPN network unit 411 and a segmentation unit 404. The feature extraction unit 401 is used to extract feature information 402 from the RGB image, the RolAlign layer 403 is used to align the feature information 402 with the RGB image, the RPN network unit 411 is used to identify the feature information 402 corresponding to the target object in the feature information 402 and transmit the feature information 402 corresponding to the target object to the RolAlign layer 403, and the segmentation unit 404 is used to segment the mask information 405 of the target object in the feature information 402.

[0047] It can be understood that after the image information of the RGB image of the target area enters the instance segmentation module 40, the instance segmentation module 40 will process the RGB image. First, all feature information 402 of the RGB image is extracted by the feature extraction unit 401 in the instance segmentation module 40, and the extracted feature information 402 is transmitted to the RolAlign layer 403 and the RPN network unit 411 respectively. The RPN network unit 411 is used to identify the feature information 402 corresponding to the target object in all the feature information 402 and transmit it to the RolAlign layer 403. At this time, the RolAlign layer 403 aligns the feature information 402 with the RGB image. In this way, the feature information 402 of the target object can be fine-tuned so that it can be more compatible with the RGB image. The feature information 402 corresponding to the identified target object and the aligned feature information 402 interact in the RolAlign layer 403, making the feature information 402 corresponding to the target object more prominent. Then, the aligned feature information 402 and the feature information 402 corresponding to the target object are respectively imported into the segmentation unit 404. Then, the segmentation unit 404 obtains the mask information 405 of the target object based on the aligned feature information 402, and transmits the obtained mask information 405 to the three-dimensional positioning module 50, so that the three-dimensional positioning module 50 can receive the mask information 405 of the target object. To summarize the above process, the RGB image provided by the image preprocessing unit enters the instance segmentation module 40, and after the feature extraction unit 401 extracts features and generates an ROI (region of interest) area candidate frame, and after aligning the original image, the pixel area (that is, the mask information 405) is respectively imported into the segmentation unit 404.

[0048] The feature information 402 is equivalent to the atlas of the area where the target object is located. The RPN network unit 411 divides the atlas into several frames, then selects the frame corresponding to the target object from the frames, and transmits the frame corresponding to the target object to the RolAlign layer 403. The RolAlign layer 403 aligns the extracted atlas (feature information 402) with the RGB image (original image), adjusts the format of the extracted atlas, and then interacts with the aligned atlas in the RolAlign layer 403, and outputs them to the segmentation unit 404 respectively. The segmentation unit 404 segments the mask information 405 of the target object in the atlas.

[0049] See also Figure 2In this embodiment, the instance segmentation module 40 also includes a fully connected layer 406 and a target detection unit 407. The fully connected layer 406 is used to convolutionally sample the aligned feature information 402 to generate a corresponding feature vector and output it to the target detection unit 407. The target detection unit 407 calculates the pixel coordinate position 408 of the target object based on the feature vector and outputs the pixel coordinate position 408 to the three-dimensional positioning module 50.

[0050] See also Figure 2 In this embodiment, the instance segmentation module 40 further includes a classification unit 409, which is configured to calculate the classification result of the target object based on the feature vector. The fully connected layer 406 outputs the feature vector generated by the convolution sampling of the aligned feature information atlas to the classification unit 409, and the classification unit 409 calculates the classification result 410 of the target object based on the feature vector. The classification result 409 obtained by the classification unit 409 acts on the segmentation unit 404 and the target detection unit 407 at the same time, so that the classification result 410 enables the visual guidance system to identify the target object to be grasped, and then grasp the target object to be grasped.

[0051] In summary, through the instance segmentation module 40, the instance segmentation module 40 adopts recognition segmentation and other technologies, so that the vision guidance system can identify the target grasped object more flexibly and more robustly.

[0052] See also Figure 3In this embodiment, the three-dimensional positioning module 50 includes a point cloud segmentation unit 501, a point cloud registration unit 502 and a template point cloud unit 504. The point cloud segmentation unit 501 extracts the spatial point cloud of the target object in the depth map according to the mask information 405 of the target object. The point cloud registration unit 502 matches the spatial point cloud with the multiple template point clouds transmitted by the template point cloud unit 504, and selects the template point cloud with the highest similarity among the multiple template point clouds for ICP calculation, and performs matrix multiplication on the calculated value and the template point cloud with the highest similarity to obtain the spatial posture of the target object. The depth map and the mask information 405 of the target object are transmitted to the point cloud segmentation unit 501. The point cloud segmentation unit 501 extracts the spatial point cloud of the target object from the depth map through the mask information 405 of the target object. The spatial point cloud of the target object is matched with the template point cloud in the template point cloud module 504, and the template point cloud with the highest similarity in the template point cloud module 504 is selected for ICP calculation. The calculated value is matrix multiplied with the spatial pose of the template point cloud to obtain the spatial pose 503 of the target object, and the spatial pose 503 of the target object is transmitted to the execution module 60. The execution module 60 guides the mobile grasping device to accurately grasp the target object according to the spatial pose 503 of the target object. Through the above process, after the target's image data enters the three-dimensional positioning module 50, the target's spatial position 503 information, namely x, y, z, rx, ry, and rz, can be calculated. The three-dimensional positioning module 50 then transmits the target's spatial position information 503 to the execution module 60, which accurately grasps the target based on the target's spatial position information 503. This vision guidance system differs from traditional machine vision solutions because it can more accurately determine the target's spatial position 503. During flexible production lines, different targets can be spatially positioned by simply using different deep learning network weights and target point cloud templates.

[0053] See also Figure 3In this embodiment, the visual guidance system also includes a posture optimization module 70, and the posture optimization module 70 includes a target tracking unit 701 and a posture graph optimization unit 702. The target tracking unit 701 is used to obtain all spatial postures of the target object transmitted by the three-dimensional positioning module 50 before the execution unit 602 grasps the target object, and calculate the spatial posture 503 information of the target object based on all spatial postures of the target object. The calculated information of all spatial postures of the target object is transmitted to the posture graph optimization unit 702, and the posture graph optimization unit 702 is used to optimize the spatial posture 503 information of the target object, and continuously and dynamically optimize and update the final grasping posture of the execution unit 602. The target tracking unit 701 is used to calculate the spatial pose 503 of the target object corresponding to all RGB images in a time sequence from the start of grasping the target object to the time before the execution unit 602 grasps the target object, and transmit the spatial pose 503 information of the target object in this time sequence to the pose graph optimization unit 702. The pose graph optimization unit 702 establishes a pose graph optimization plan based on the spatial pose 503 of the target object in this time sequence, and dynamically optimizes the final grasping pose.

[0054] See also Figure 1 In this embodiment, the visual guidance system for a mobile grasping device further includes a calibration module 10. The calibration module 10 is used to calibrate the execution unit 602 and the visual guidance system, and to place the execution unit 602 and the guidance system in a standby state, thereby returning the execution unit 602 and the guidance system to a "zero" state. This allows for faster adaptation from scratch when the execution unit 602 and the visual guidance system are adapted. The execution unit 602 can be considered the grasping end of the grasping device.

[0055] The calibration module 10 performs hand-eye calibration based on the single lens of the binocular camera, and simultaneously obtains the relative relationship between the binocular camera and the end of the mobile grasping device (camera extrinsic parameters) such as the pixel center point and distortion parameters.

[0056] See also Figure 1 In this embodiment, the visual guidance system for a mobile grasping device further includes a trigger module 20, which is configured to trigger the image acquisition and preprocessing module 30 to acquire and preprocess the image data. When it is necessary to grasp an object, the image acquisition and preprocessing module 30 is triggered by the trigger 20, enabling the image acquisition and preprocessing module 30 to quickly respond and acquire image data of the target area.

[0057] This vision guidance system can be used on composite mobile robots, overhead cranes and other production line equipment, enabling composite robots and overhead cranes to grasp targets more accurately when working on the production line. In addition, the vision guidance system and the grasping equipment do not require long-term adaptation before they can be put into use, effectively saving time costs.

[0058] When the vision guidance system is used on a composite mobile robot, the vision camera is mounted on the end of the robotic arm, and the vision system is deployed on the vision controller and placed inside the composite mobile robot, communicating with the composite robot through Ethernet networking and ROS messages.

[0059] When the vision guidance system is used on the overhead crane mobile grasping device, the vision camera is installed at the end of the overhead crane fixture, overlooking the target object. The vision system is deployed on the vision controller, which is placed in the external electrical control cabinet of the overhead crane mobile grasping device system. It communicates with the on-site PCL through the 458 bus and with the host computer through Ethernet.

[0060] In summary, the visual guidance system for a mobile grasping device in the above-mentioned embodiment of the present invention collects image data of the target area through the data acquisition and preprocessing module 30 and preprocesses the collected image data, and then transmits the preprocessed image information to the instance segmentation module 40. The instance segmentation module 40 outputs the mask information 405 and the pixel coordinate position 408 of the image, and then transmits the mask information 405 and the pixel coordinate position 408 to the three-dimensional positioning module 50. The three-dimensional positioning module 50 calculates the spatial pose 503 of the target object through the mask information 405 and the pixel coordinate position 408, and then transmits the spatial pose 503 of the target object to the execution module 60. The execution module 60 can accurately grasp the target object according to the guidance of the spatial pose 503 information of the target object. Since the visual guidance system can accurately calculate the spatial pose 503 of the target object, the time for adapting the visual guidance system to the mobile grasping device is short, and the mobile grasping device can be directly put into use, saving time costs on the production line.

[0061] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0062] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A visual guidance system for a mobile grasping device, characterized in that: It includes data acquisition and preprocessing module, instance segmentation module, 3D positioning module and execution module; The data acquisition and preprocessing module is used to acquire image data, preprocess the acquired image data, and transmit the preprocessed image data to the instance segmentation module; The instance segmentation module is used to identify and segment the target object in the preprocessed image data, and output the pixel coordinate position and mask information of the target object to the three-dimensional positioning module; The three-dimensional positioning module is used to receive the pixel coordinate position and mask information of the target object, calculate the spatial position of the target object, and transmit the spatial position information of the target object to the execution module; The execution module is used to receive the spatial posture information of the target object and grasp the target object according to the spatial posture information of the target object; The point cloud segmentation unit extracts the spatial point cloud of the target object in the depth map according to the mask information of the target object. The point cloud registration unit matches the spatial point cloud with the multiple template point clouds transmitted by the template point cloud unit, selects the template point cloud with the highest similarity among the multiple template point clouds for ICP calculation, and performs matrix multiplication on the calculated value and the template point cloud with the highest similarity to obtain the spatial pose of the target object. The visual guidance system further includes a posture optimization module, which includes a target tracking unit and a posture graph optimization unit; The target tracking unit is used to obtain all spatial poses of the target object transmitted by the three-dimensional positioning module before the execution unit grasps the target object, and calculate the spatial pose information of the target object based on all spatial poses of the target object; The pose graph optimization unit is used to optimize the spatial pose information of the target object to obtain the final grasping pose of the execution unit. The pose graph optimization unit establishes a pose graph optimization scheme based on the spatial pose of the target object within this time sequence and dynamically optimizes the final grasping pose.

2. The visual guidance system for a mobile grasping device according to claim 1, characterized in that: The execution module includes a logic controller and an execution unit. The logic controller is used to receive spatial posture information of the target object and control and guide the execution unit to perform a grasping operation.

3. The visual guidance system for a mobile grasping device according to claim 2, characterized in that: The data acquisition and preprocessing module includes an image acquisition unit and a data preprocessing unit, wherein the image acquisition unit is used to acquire the image data and transmit the acquired image data to the data preprocessing unit, wherein the image data includes at least an RGB image and a depth image; The data preprocessing unit is used to preprocess the RGB image and the depth image, and transmit the preprocessed RGB image to the instance segmentation module, and transmit the preprocessed depth map to the three-dimensional positioning module.

4. The visual guidance system for a mobile grasping device according to claim 3, characterized in that: The instance segmentation module includes a feature extraction unit, a RolAlign layer, an RPN network unit and a segmentation unit; The feature extraction unit is used to extract feature information from the RGB image; The RolAlign layer is used to align the feature information with the RGB image; The RPN network unit is used to identify the feature information corresponding to the target object in the feature information, and transmit the feature information corresponding to the target object to the RolAlign layer; The segmentation unit is used to segment the mask information of the target object in the feature information.

5. The visual guidance system for a mobile grasping device according to claim 4, characterized in that: The instance segmentation module also includes a fully connected layer and a target detection unit. The fully connected layer is used to convolutionally sample the aligned feature information to generate a corresponding feature vector and output it to the target detection unit. The target detection unit calculates the pixel coordinate position of the target object based on the feature vector and outputs the pixel coordinate position to the three-dimensional positioning module.

6. The visual guidance system for a mobile grasping device according to claim 5, characterized in that: The instance segmentation module further includes a classification unit, which is configured to calculate a classification result of the target object based on the feature vector.

7. The visual guidance system for a mobile grasping device according to claim 1, characterized in that: The vision guidance system further includes a calibration module, which is used to calibrate the execution unit and the vision guidance system.

8. The visual guidance system for a mobile grasping device according to claim 1, characterized in that: The visual guidance system further includes a trigger module, which is used to trigger the image acquisition and preprocessing module to enable the image acquisition and preprocessing module to acquire the image data and preprocess the image data.

Images