Visual mechanical arm article positioning and picking method and system

Abstract

The invention relates to the technical field of artificial intelligence vision and mechanical arm application, in particular to a visual mechanical arm article positioning and picking method and system, which integrates high-precision visual detection and a flexible mechanical arm execution mechanism, realizes a full-automatic process from target detection to precise picking, reduces the burden of manual operation, and improves the working efficiency. Compared with the prior art, the method has the advantages that the production efficiency is improved, the deep lightweight improvement is carried out aiming at the pain points of large calculation amount and high resource consumption of a traditional visual recognition system, and the performance of the system is multiplied compared with that of an original model by optimizing an algorithm structure and reducing the model parameter quantity and the calculation complexity; and meanwhile, the dependence on computing resources and hardware configuration is remarkably reduced, so that the system can efficiently operate on low-cost hardware platforms such as Raspberry Pi, the deployment cost is greatly reduced, and the popularity rate and usability of the system are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence vision and robotic arm application technology, and in particular to a method and system for positioning and picking objects by a visual robotic arm. Background Art

[0002] With the rapid development of automation and intelligent technology, visual robotic arm systems have shown great application potential in many fields such as industrial production, home services and scientific research due to their high efficiency and flexibility. Especially in the scene of picking and sorting objects, the visual robotic arm system has significantly improved the work efficiency, reduced manual intervention and reduced labor intensity with its high precision and high efficiency. However, most traditional visual robotic arm systems rely on high-performance computers or servers to complete image processing and decision-making tasks. This feature greatly limits its wide application in scenarios with limited resources and high portability requirements. Summary of the invention

[0003] In view of this, an object of the present invention is to propose a method and system for positioning and picking objects with a visual robotic arm, so as to solve the problem that the existing visual robotic arm system is inconvenient to use in scenarios with limited resources and high portability requirements.

[0004] Based on the above purpose, the present invention provides a method for positioning and picking objects by a visual robot arm, comprising the following steps:

[0005] S1. Collect a video of the object to be detected, divide the video into images by frame, and mark the object to be detected in the image, where the marking includes a mark to be picked up and the object to be picked up;

[0006] S2, perform data enhancement processing on the annotated images to obtain an image dataset, and divide the image dataset into a training set, a validation set, and a test set according to a certain ratio;

[0007] S3. Train the YOLOv5-pico model using the training set, and iteratively optimize the model parameters until the model performance meets the requirements to obtain the YOLOv5-pico inference model, wherein the YOLOv5-pico model is obtained by lightweighting the YOLOv5 model by reducing the width ratio hyperparameter;

[0008] S4. Deploy the trained YOLOv5-pico inference model to the Raspberry Pi, configure the image acquisition module and robotic arm, and perform hand-eye calibration.

[0009] S5, perform real-time detection of the target to be detected through the YOLOv5-pico inference model to obtain the target category and pixel coordinates;

[0010] S6. Based on the results of real-time detection, if a mark or object to be picked up is detected, the position of the target in the robot arm coordinate system is calculated based on the pixel target, and the robot arm is controlled to perform a picking operation.

[0011] Preferably, the positioning and picking method further includes:

[0012] S7, after completing the picking operation of the robotic arm, returning the robotic arm to the initial position or moving it to the next preset position;

[0013] S8, repeat steps S6 and S7, continue to wait and detect the appearance of the to-be-picked mark to perform the next round of picking operation, and realize closed-loop unmanned control.

[0014] Preferably, the improvement of the YOLOv5-pico model also includes:

[0015] The Bottleneck module in the original C3 module of the YOLOv5 model is replaced with a universal inverted bottleneck module. Inside the inverted bottleneck module, the point-by-point convolution layer is first used to effectively compress the channel dimension of the input data. Then the deep convolution layer is used as the core processing unit to act independently on each channel. Finally, the point-by-point convolution layer is used again to reconstruct the dimension and integrate the channels of the feature map.

[0016] Preferably, the data enhancement processing includes but is not limited to image cropping, rotation, noise introduction, and adjustment of brightness, contrast, and saturation.

[0017] Preferably, performing real-time detection on the target to be detected includes:

[0018] The video stream is captured by a camera placed at the end of the robotic arm, and the image capture parameters are preset to reduce the image size;

[0019] The frame skipping technology is introduced to preset a specific number of frames, and the filtered frames are input into the YOLOv5-pico network to achieve fast and accurate target detection, including the identification of target categories and their pixel coordinates.

[0020] Preferably, calculating the position of the target in the robot arm coordinate system according to the pixel target and controlling the robot arm to perform the picking operation includes:

[0021] When the preset mark to be picked up is detected, the robot arm is adjusted to the working area, ready to enter the next stage of the workflow and start the secondary inspection process for the target object;

[0022] After successfully detecting the object to be picked up, the integrated coordinate conversion algorithm converts the pixel coordinate information obtained during the image recognition process into physical space coordinates executable by the robot arm in real time, thereby accurately guiding the robot arm to move to the specific location of the object to be picked up;

[0023] After the robotic arm reaches the predetermined position, it performs a picking operation to complete the task of grabbing or moving the target object.

[0024] Preferably, the hand-eye calibration includes:

[0025] Establish the precise correspondence between pixel coordinates and robot arm coordinates;

[0026] Through the corresponding relationship, the one-dimensional linear interpolation technology is used to accurately map the pixel coordinates to be converted into the robot coordinate system to obtain the robot coordinates.

[0027] The present invention also provides a visual robot arm object positioning and picking system, comprising:

[0028] An image acquisition module, used to acquire video of the object to be detected;

[0029] The image processing module is used to divide the video into images by frame, annotate the objects to be detected in the images, and perform data enhancement processing on the annotated images to obtain an image data set, and divide the image data set into a training set, a validation set, and a test set according to a certain ratio;

[0030] The model training module is used to train the YOLOv5-pico model using the training set, and obtain the YOLOv5-pico inference model by iteratively optimizing the model parameters until the model performance meets the requirements. The YOLOv5-pico model is obtained by lightweighting the YOLOv5 model by reducing the width ratio hyperparameter.

[0031] The deployment module is used to deploy the trained YOLOv5-pico inference model to the Raspberry Pi, configure the image acquisition module and the robotic arm, and perform hand-eye calibration;

[0032] Real-time detection module, used to detect the target in real time through YOLOv5-pico inference model, and obtain the target category and pixel coordinates;

[0033] The control module is used to calculate the position of the target in the robot arm coordinate system based on the pixel target according to the result of real-time detection, and control the robot arm to perform the picking operation if a mark or object to be picked up is detected.

[0034] Beneficial effects of the present invention:

[0035] (1) High degree of automation, greatly liberating human resources: This system integrates high-precision visual inspection modules and flexible robotic arm actuators to achieve a fully automated process from target detection to precise picking. This revolutionary design greatly reduces the burden of manual operation, improves production efficiency, and reduces the error rate caused by human factors, saving a lot of human resource costs for enterprises;

[0036] (2) Lightweight design to reduce computing and hardware costs: To address the pain points of traditional visual recognition systems, such as large computational complexity and high resource consumption, the present invention has made in-depth lightweight improvements. By optimizing the algorithm structure and reducing the number of model parameters and computational complexity, the system has achieved exponential improvement in performance compared to the original model, while significantly reducing the dependence on computing resources and hardware configuration. This not only enables the system to run efficiently on low-cost hardware platforms such as the Raspberry Pi, but also significantly reduces deployment costs, thereby improving the popularity and availability of the system.

[0037] (3) Improve real-time performance and enhance system responsiveness: Lightweight design not only reduces computing costs, but also directly improves the real-time performance of the system. In a rapidly changing industrial environment, the system can respond quickly, capture and process image information in real time, accurately identify target objects, and guide the robot arm to pick them up accurately. This efficient real-time processing capability ensures the continuity and stability of the production line and improves overall production efficiency;

[0038] (4) High practical application value, widely applicable to various scenarios: The robot arm visual inspection and picking system proposed in this invention has extremely high practical application value due to its high efficiency, intelligence and low cost. It is not only applicable to traditional manufacturing fields, such as parts assembly and material handling, but can also be expanded to multiple industries such as warehousing and logistics, agricultural picking and medical assistance, providing strong technical support for automated operations in different scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 This is a flow chart of a method for locating and picking up items according to an embodiment of the present invention;

[0041] Figure 2 This is a detection effect diagram of the test set of an embodiment of the present invention;

[0042] Figure 3 This is a diagram of the improved yolov5-pico network structure of an embodiment of the present invention;

[0043] Figure 4 A schematic diagram of the C3_UIB structure of an embodiment of the present invention;

[0044] Figure 5 It is a schematic diagram of the structure of the bottleneck inverting module UIB according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments.

[0046] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0047] like Figure 1 As shown, the embodiment of this specification provides a method for positioning and picking objects by a visual robot arm, comprising the following steps:

[0048] Step 1: This method first uses a high-resolution camera to capture a video stream, and then intelligently divides the continuous video data into a series of high-definition and independent image frame sequences through precisely set frame rate parameters. This process accurately captures the key morphological features and dynamic details of the object, building a rich and reliable visual data foundation for subsequent analysis and recognition.

[0049] Step 2: Use image annotation software to manually annotate the targets in the selected image frames. The annotations include the signs and items to be picked up, ensuring accuracy and detailed structured information. This process creates a comprehensive label file system that records information such as target location, size, and category in detail, and improves the semantic level of image data, providing solid support for subsequent image analysis and intelligent recognition.

[0050] Step 3: To further improve the generalization ability, prediction accuracy, and robustness of the machine learning model under variable input conditions, apply data augmentation algorithms to the accurately annotated images and their corresponding labels;

[0051] This strategy effectively expands the size of the dataset by automatically transforming image data (such as rotating, scaling, cropping, color adjustment, and introducing noise), while retaining the core features of the original data, providing more diverse training samples for model learning. This process enhances the adaptability of the model and significantly improves its performance in unknown or complex scenarios.

[0052] Then, the data is divided into training set, validation set and test set according to scientific proportions to ensure that each set is representative, meets training requirements and supports performance verification and evaluation, such as Figure 2 The detection effect diagram of this embodiment in the test set is given.

[0053] Step 4: When exploring how to adapt the target detection algorithm to the resource-limited Raspberry Pi device, this embodiment carefully designs a lightweight and improved version of the YOLOv5 algorithm: YOLOv5-pico, the network structure is as follows Figure 3 As shown in Figure 2, this innovation reduces the number of network parameters and computational complexity by multiples, overcoming the limitations of low computing power environments while maintaining high accuracy and efficiency.

[0054] It should be noted that two key optimizations were implemented in the lightweight improved YOLOv5-pico algorithm: first, an innovative tiny lightweight improved model was proposed, which greatly reduced the computational complexity and memory usage of the model by streamlining the network structure and optimizing parameter configuration, thus achieving a truly lightweight design; second, an advanced inverted bottleneck module was introduced, which can maintain the original function while being lightweight, providing strong support for target detection tasks in low computing power environments;

[0055] The tiny lightweight improved model, namely YOLOv5-pico, is an innovatively designed new lightweight improved model, which is specially designed to solve the network redundancy problem generated by the traditional general target detection model when processing single or a few category tasks. It cleverly balances the relationship between detection accuracy and model complexity. Through precise optimization strategies, it achieves a significant lightweight effect while maintaining high-precision detection capabilities, thereby effectively reducing deployment costs and resource consumption during operation.

[0056] Specifically, the essence of YOLOv5-pico's lightweight design lies in reducing the network width scaling factor. By introducing and fine-tuning the key hyperparameter of the width scaling factor (width_multiple), the model can reduce the number of convolution kernels and output channels in each layer. This process not only maintains the model's effective ability to extract target features, but also significantly reduces the model's computational complexity and total number of parameters. This strategy not only enables YOLOv5-pico to run smoothly on devices with limited resources, but also greatly improves its flexibility and adaptability in actual application scenarios, demonstrating its outstanding value as a lightweight target detection solution. The comparison of network scaling parameters of YOLOv5-pico and YOLOv5n is shown in Table 1.

[0057] Table 1 Comparison of network scaling parameters between YOLOv5-pico and YOLOv5n

[0058]

[0059] The inverted bottleneck module, while retaining the C3 module as its core architecture, has made a key innovation: the original Bottleneck module has been upgraded and replaced with a more efficient Universal Inverted Bottlenecks (UIB).

[0060] The C3 module builds an efficient and sophisticated feature processing pipeline. The input feature map first passes through a carefully designed initial convolution layer, which quickly captures and extracts the basic feature contours in the input data. Subsequently, these preliminary refined feature maps are sent to the Bottleneck module, which further refines more complex and abstract feature expressions with its unique deep mining capabilities.

[0061] When deeply processing features, the C3 module also retains the pure details of the original feature map and the intuitive information of the input data. When fusing features, C3 cleverly splices these original features with the deep features of Bottleneck, achieving cross-level intersection and complementarity of feature information, thereby producing a richer and more comprehensive feature expression.

[0062] In order to further optimize the model architecture, the Bottleneck module in C3 is fully upgraded to the UIB module. This measure aims to reduce the number of model parameters and computational complexity from the root, while improving the efficiency and accuracy of feature extraction. Figure 4 as shown.

[0063] Inside the UIB module, the input data is first compressed in the channel dimension through an efficient point-by-point convolution layer. This step reduces a lot of parameter calculations, lightens the burden for subsequent processing, and also leaves room for in-depth exploration of feature extraction.

[0064] Subsequently, the UIB module introduced a deep convolution layer as the core processing unit. This innovation replaced the traditional 3x3 convolution. With its unique parallel processing capability, deep convolution acts independently on each channel, which not only ensures the quality of feature extraction, but also greatly reduces the parameter requirements, completing the feature extraction task in a more efficient way, making the model more capable of processing large-scale or complex data.

[0065] Finally, the UIB module uses the point-by-point convolution layer again to reconstruct the dimension and integrate the channels of the feature map. This process brings together the scattered feature information, making them more practical and ensuring that the final extracted features are both rich and diverse and highly condensed, providing a solid and reliable foundation for the subsequent decision-making of the model. The UIB structure diagram is shown below: Figure 5 shown.

[0066] Step 5: Input the dataset into the YOLOv5-pico network and train it on a high-performance computer or server to generate an efficient inference model.

[0067] Step 6: When the system is started and put into operation, the robotic arm controlled by the Raspberry Pi first performs the initialization steps autonomously, moves accurately to the preset starting position, and enters the working state;

[0068] Next, the Raspberry Pi's camera is turned on, and its image capture parameters are carefully preset to reduce the image read size while still fully retaining the key information required for detection. This greatly reduces the pressure of data processing without seriously sacrificing accuracy, and significantly improves the instant response capability of detection.

[0069] In the crucial target detection link, this embodiment cleverly integrates the frame skipping processing method. The system can screen the continuous video stream and only transmit the representative frames that meet the preset frame number threshold to YOLOv5-pico for detection, which greatly reduces the data processing burden of the system and improves the real-time performance.

[0070] These carefully selected frames are then passed to the lightweight YOLOv5-pico network that has been deeply optimized and successfully deployed for the Raspberry Pi environment. With its excellent performance and lightweight design, the network can complete fast and accurate target detection analysis of the input frames in a short time, and accurately identify the category of the target object and its corresponding pixel coordinates.

[0071] Step 7: Once the system detects the preset mark to be picked up, the robot arm responds immediately, automatically adjusts to the working area, enters the next stage of work, and starts the secondary inspection process for the target object;

[0072] Step 8: When the system successfully identifies the object to be picked up, it uses a coordinate conversion algorithm based on the pre-completed hand-eye calibration results, that is, the precise correspondence between pixel coordinates and robotic arm coordinates. Through a one-dimensional linear interpolation method, the pixel coordinates detected in the image are accurately mapped to the robotic arm coordinates. This process ensures that the robotic arm can move accurately to the location of the object to be picked up and successfully perform the picking operation.

[0073] Step 9: After completing the picking task, the robot arm automatically returns to the initial position and waits for the next detection signal of the mark to be picked up.

[0074] This embodiment also provides a visual robot arm object positioning and picking system, including:

[0075] An image acquisition module, used to acquire video of the object to be detected;

[0076] The image processing module is used to divide the video into images by frame, annotate the objects to be detected in the images, and perform data enhancement processing on the annotated images to obtain an image data set, and divide the image data set into a training set, a validation set, and a test set according to a certain ratio;

[0077] The model training module is used to train the YOLOv5-pico model using the training set, and obtain the YOLOv5-pico inference model by iteratively optimizing the model parameters until the model performance meets the requirements. The YOLOv5-pico model is obtained by lightweighting the YOLOv5 model by reducing the width ratio hyperparameter.

[0078] The deployment module is used to deploy the trained YOLOv5-pico inference model to the Raspberry Pi, configure the image acquisition module and the robotic arm, and perform hand-eye calibration;

[0079] Real-time detection module, used to detect the target in real time through YOLOv5-pico inference model, and obtain the target category and pixel coordinates;

[0080] The control module is used to calculate the position of the target in the robot arm coordinate system based on the pixel target according to the result of real-time detection, and control the robot arm to perform the picking operation if a mark or object to be picked up is detected.

[0081] In summary, the core highlight of this embodiment is that the YOLOv5 neural network is deeply optimized and lightweighted, and a lightweight version YOLOv5-pico is proposed. While maintaining excellent target detection accuracy, the algorithm achieves a significant reduction in model parameters and computational complexity, greatly broadens its application potential on resource-constrained edge computing devices (such as Raspberry Pi), and significantly improves operating efficiency and real-time response capabilities. Relying on this innovative algorithm, the present invention constructs a set of fully automated object positioning and picking methods, which can independently complete the entire process from target detection to precise positioning, and then to object picking without human intervention, which not only greatly improves operating efficiency and accuracy, but also demonstrates a very high level of intelligence. This technological innovation has injected vitality into the fields of industrial automation and intelligent warehousing, achieved significant efficiency improvements and technological leaps, built a solid technical foundation and a strong driving force for these industries, and led the related industries to steadily move towards a new chapter of greater efficiency, intelligence and unmanned operation.

[0082] In order to demonstrate the practical effect of the algorithm proposed in this invention, a set of ablation experiments are designed as shown in Table 2.

[0083] Table 2 Ablation experiment results

[0084]

[0085] From the clear presentation of the above table, we can clearly see that the number of parameters, the amount of calculation, and the model size have all been significantly reduced by multiples. This optimization greatly reduces the difficulty of deployment on edge devices, not only significantly reducing the required computing resources, but also effectively reducing hardware costs. For devices with relatively limited computing power such as Raspberry Pi, this improvement is particularly critical, providing strong support for the efficient operation of complex machine learning models, which is of great significance.

[0086] Those skilled in the art should understand that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the present invention is limited to these examples; under the concept of the present invention, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes in different aspects of the present invention as described above, which are not provided in detail for the sake of simplicity. Any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for positioning and picking objects by a visual robot arm, characterized in that: The method comprises the following steps: S1. Collect a video of the object to be detected, divide the video into images by frame, and mark the object to be detected in the image, wherein the marking includes a mark to be picked up and the object to be picked up; S2, perform data enhancement processing on the annotated images to obtain an image dataset, and divide the image dataset into a training set, a validation set, and a test set according to a certain ratio; S3. Train the YOLOv5-pico model using the training set, and iteratively optimize the model parameters until the model performance meets the requirements to obtain the YOLOv5-pico inference model, wherein the YOLOv5-pico model is obtained by lightweighting the YOLOv5 model by reducing the width ratio hyperparameter; S4. Deploy the trained YOLOv5-pico inference model to the Raspberry Pi, configure the image acquisition module and robotic arm, and perform hand-eye calibration. S5, perform real-time detection of the target to be detected through the YOLOv5-pico inference model to obtain the target category and pixel coordinates; S6. According to the result of real-time detection, if a mark or object to be picked up is detected, the position of the target in the robot arm coordinate system is calculated based on the pixel target, and the robot arm is controlled to perform a picking operation.

2. The method for positioning and picking objects by a visual robot arm according to claim 1, characterized in that: The positioning and picking method also includes: S7, after completing the picking operation of the robotic arm, returning the robotic arm to the initial position or moving it to the next preset position; S8, repeat steps S6 and S7, continue to wait and detect the appearance of the to-be-picked mark to perform the next round of picking operation, and realize closed-loop unmanned control.

3. The method for positioning and picking objects by a visual robot arm according to claim 1, characterized in that: Improvements to the YOLOv5-pico model also include: The Bottleneck module in the original C3 module of the YOLOv5 model is replaced with a universal inverted bottleneck module. Inside the inverted bottleneck module, the point-by-point convolution layer is first used to effectively compress the channel dimension of the input data. Then the deep convolution layer is used as the core processing unit to act independently on each channel. Finally, the point-by-point convolution layer is used again to reconstruct the dimension and integrate the channels of the feature map.

4. The method for positioning and picking objects by a visual robot arm according to claim 1, characterized in that: The data enhancement processing includes but is not limited to image cropping, rotation, noise introduction, and adjustment of brightness, contrast, and saturation.

5. The method for positioning and picking objects by a visual robot arm according to claim 1, characterized in that: The real-time detection of the target to be detected includes: The video stream is captured by a camera placed at the end of the robotic arm, and the image capture parameters are preset to reduce the image size; The frame skipping technology is introduced to preset a specific number of frames, and the filtered frames are input into the YOLOv5-pico network to achieve fast and accurate target detection, including the identification of target categories and their pixel coordinates.

6. The method for positioning and picking objects by a visual robot arm according to claim 1, characterized in that: Calculating the position of the target in the robot arm coordinate system according to the pixel target and controlling the robot arm to perform the picking operation includes: When the preset mark to be picked up is detected, the robot arm is adjusted to the working area, ready to enter the next stage of the workflow and start the secondary inspection process for the target object; After successfully detecting the object to be picked up, the integrated coordinate conversion algorithm converts the pixel coordinate information obtained during the image recognition process into physical space coordinates executable by the robot arm in real time, thereby accurately guiding the robot arm to move to the specific location of the object to be picked up; After the robotic arm reaches the predetermined position, it performs a picking operation to complete the task of grabbing or moving the target object.

7. The method for positioning and picking objects by a visual robot arm according to claim 1, characterized in that: The hand-eye calibration includes: Establish the precise correspondence between pixel coordinates and robot arm coordinates; Through the corresponding relationship, the one-dimensional linear interpolation technology is used to accurately map the pixel coordinates to be converted into the robot coordinate system to obtain the robot coordinates.

8. A visual robot arm object positioning and picking system, characterized in that: include: An image acquisition module, used to acquire video of the object to be detected; The image processing module is used to divide the video into images by frame, annotate the objects to be detected in the images, and perform data enhancement processing on the annotated images to obtain an image data set, and divide the image data set into a training set, a validation set, and a test set according to a certain ratio; The model training module is used to train the YOLOv5-pico model using the training set, and obtain the YOLOv5-pico inference model by iteratively optimizing the model parameters until the model performance meets the requirements. The YOLOv5-pico model is obtained by lightweighting the YOLOv5 model by reducing the width ratio hyperparameter. The deployment module is used to deploy the trained YOLOv5-pico inference model to the Raspberry Pi, configure the image acquisition module and the robotic arm, and perform hand-eye calibration; Real-time detection module, used to detect the target in real time through YOLOv5-pico inference model, and obtain the target category and pixel coordinates; The control module is used to calculate the position of the target in the robot arm coordinate system based on the pixel target according to the result of real-time detection, and control the robot arm to perform the picking operation if a mark or object to be picked up is detected.

Images