Dynamic scanning positioning and grabbing system and method based on j1 axis servo vision

Abstract

This disclosure relates to the fields of industrial automation and machine vision technology, and in particular to a dynamic scanning positioning and grasping system and method based on J1-axis follow-up vision. The system includes: a robotic arm having a J1 axis and a rotating base that rotates synchronously with the J1 axis; an end effector mounted at the end of the robotic arm for grasping a workpiece; a follow-up vision assembly including a global vision unit whose field of view covers the working area of ​​the end effector; and a control assembly communicatively connected to the robotic arm, the end effector, and the global vision unit. This solution addresses the problems in existing technologies such as difficulty in adapting to non-standard sized sheet metal, high reflective interference, high height measurement costs, low operating efficiency, and unstable long-term grasping accuracy.

Description

Technical Field

[0001] This disclosure relates to the fields of industrial automation and machine vision technology, and in particular to a dynamic scanning positioning and grasping system and method based on J1-axis servo vision. Background Technology

[0002] In existing custom furniture production, the handling of various non-standard sized rectangular panels typically relies on manual labor or traditional gantry-type vision systems. Existing technologies suffer from several unresolved pain points. First, identifying stacked panels of different sizes is difficult. Custom furniture panels have a wide size range, from 200mm to 2440mm in length and 50mm to 1220mm in width. Skewed stacking, variations in height, or visual interference from different material colors can all cause fixed vision systems to fail. Second, identifying high-gloss and reflective panels is challenging. For mirrored panels or panels with different reflective properties, fixed-angle vision units easily capture ambient light or reflected light spots, resulting in overexposed images or feature loss, making it impossible to accurately extract panel edges. Furthermore, the development and deployment costs of vision systems are high. Third, height measurement and panel boundary measurement are costly. To obtain the stacking height of the top layer of panels, expensive large-field-of-view, high-precision 3D structured light cameras or laser profilometers are usually required. Ordinary 2D cameras cannot determine height information from a single viewpoint, and economical 3D depth vision units struggle to achieve millimeter-level positioning accuracy in viewing areas exceeding 3 meters.

[0003] Visual inspection and robotic arm operations follow a sequential process. The traditional workflow involves robotic arm positioning, camera taking pictures, coordinate calculation, and robotic arm grasping. Visual inspection consumes a significant amount of robotic arm waiting time, making it difficult to improve production cycle time. Long-term operation of the robotic arm will generate zero-point drift and cumulative errors, and relying solely on single positioning is insufficient to guarantee stable grasping accuracy over a long period. Summary of the Invention

[0004] This disclosure provides a dynamic scanning positioning and grasping system and method based on J1-axis servo vision. It constructs a rigid, high-position global perception and end-effector local closed-loop correction robotic arm recognition and grasping architecture, solving the problems of existing technologies such as difficulty in adapting to non-standard sized materials, significant reflective interference, high height measurement costs, low operating efficiency, and unstable long-term grasping accuracy. The technical solution is as follows:

[0005] In a first aspect, embodiments of this disclosure provide a dynamic scanning, positioning, and grasping system based on J1-axis homing vision, comprising:

[0006] A robotic arm having a J1 axis and a rotating base that rotates synchronously with the J1 axis;

[0007] An end effector, which is mounted at the end of a robotic arm for gripping a workpiece;

[0008] The follow-up vision component includes a global vision unit, the field of view of which covers the working area of ​​the end effector;

[0009] The control component, which is communicatively connected to the robotic arm, end effector, and global vision unit, is configured to:

[0010] The J1 axis is driven to rotate, thereby enabling the global vision unit to perform multi-angle scanning of the workpiece stack. Multi-view image fusion is used to eliminate reflection interference and complete workpiece positioning. The baseline distance formed by the rotation of the J1 axis and the principle of monocular parallax are used to calculate the height information of the workpiece. While the robotic arm is performing the handling operation, the global vision unit is controlled to synchronously complete the scanning of the remaining workpiece stack and pre-calculate the gripping pose of the next workpiece, realizing the parallel operation of handling and vision scanning.

[0011] Preferably, the global vision unit is a wide-angle industrial area array camera. The control component detects the proportion of overexposed pixels and the gradient variance of the image, selects the image frame with the least reflection and the highest clarity as the positioning image, or stitches and merges the effective areas without overexposure of multiple frames to generate a workpiece positioning image without reflection interference.

[0012] Preferably, the control component constructs a monocular stereo vision model based on the rotation angle of the J1 axis and the installation height of the global vision unit. By matching the workpiece feature points in images from different perspectives, the parallax displacement is calculated, and the height of the top surface of the workpiece stack is solved in reverse, thus realizing Z-axis coordinate measurement without the need for an additional 3D camera.

[0013] Preferably, the system further includes a local depth vision sensor mounted on the end effector. The control component first uses a global vision unit to perform large-scale coarse positioning of the workpiece, then guides the end effector to move above the gripping point, and then uses the local depth vision sensor to perform close-range fine positioning. Based on the fine positioning result, the data deviation of the coarse positioning is corrected, and the gripping pose of the workpiece is updated.

[0014] Preferably, the end effector further includes a gripping component, which is a combination of two sponge suction cups, including a wide suction cup and a narrow suction cup, capable of adsorbing workpieces individually or simultaneously depending on their size.

[0015] On the other hand, this disclosure also discloses a dynamic scanning positioning and grasping method based on J1-axis servo vision, including the following steps:

[0016] The rigid support is fixed to the base of the robotic arm, so that the global vision unit mounted on the top of the rigid support can rotate synchronously with the J1 axis.

[0017] Drive the J1 axis to reciprocate within a preset angle range, thereby driving the global vision unit to acquire multi-angle images of the workpiece stack, and remove reflection interference and identify the workpiece outline through multi-view image fusion.

[0018] Based on the baseline distance formed by the rotation of the J1 axis, the height information of the top surface of the workpiece stack is calculated using the monocular parallax principle.

[0019] During the workpiece handling operation, the J1 axis is synchronously driven to scan the remaining workpiece stack with the global vision unit and pre-calculate the gripping pose of the next workpiece.

[0020] After the robotic arm completes the unloading and returns to the picking position, it directly performs the next picking based on the pre-calculated picking posture, achieving continuous operation without waiting.

[0021] Preferably, the step of removing reflective interference through multi-view image fusion specifically includes: calculating the gradient variance and overexposed pixel ratio of each frame in real time, selecting the image frame with the least reflective interference and the highest clarity as the positioning image, or performing pixel-level stitching and fusion of the non-overexposed areas of multiple frames to generate a virtual positioning image without reflective interference.

[0022] Preferably, the step of calculating height information using the monocular parallax principle specifically includes: controlling the J1 axis to rotate to two different angles to form a measurement baseline, extracting the parallax displacement of the same workpiece feature point in images from different viewing angles, and combining the installation height of the global vision unit with the rotation angle of the J1 axis to solve for the top surface height of the workpiece stack.

[0023] Preferably, it also includes a dual positioning step: first, the workpiece is coarsely positioned over a large area by a global vision unit; after the robotic arm moves above the gripping point, fine positioning is performed at close range by a local depth vision sensor integrated into the end effector; and the data deviation of the coarse positioning and the gripping pose are corrected based on the fine positioning results.

[0024] Preferably, in the step of driving the J1 axis to reciprocate within a preset angle range, the preset angle range is ±30°, and the time taken for the robotic arm to perform the plate handling operation is 2 to 5 seconds. During this time, the scanning of the remaining plate stack and the pre-calculation of the next pose are completed.

[0025] The advantages or beneficial effects of the above technical solutions include at least the following:

[0026] First, it has strong anti-reflective interference capability. By controlling the J1 axis, the global vision unit of the follow-up vision component is driven to scan from multiple angles. Combined with multi-view image fusion, it effectively solves the recognition failure problem of high-gloss workpieces, glass workpieces, and dark matte workpieces, and stably extracts the true boundary of the workpiece.

[0027] Secondly, hardware costs are significantly reduced. By utilizing the baseline distance formed by the rotation of the J1 axis and the principle of monocular parallax, the workpiece stacking height can be measured with a common wide-angle industrial area array camera, eliminating the need for expensive 3D structured light cameras or laser profilometers, thus greatly reducing equipment deployment costs.

[0028] Third, the efficiency of operation is greatly improved. By using a parallel operation mechanism of handling and visual scanning, the visual inspection time is completely hidden within the handling cycle of the robotic arm. The theoretical visual inspection time is close to zero, which significantly improves the output per unit hour of the production line and solves the pain point of slow cycle time in traditional serial operations.

[0029] Fourth, the grasping accuracy is stable over a long period of time. It can correct the cumulative error generated by the long-term operation of the robotic arm in real time through a dual positioning mechanism of coarse positioning by global vision unit and fine positioning by end-effector local depth vision sensor, so as to ensure the grasping stability of long-term continuous operation and reduce the grasping failure rate.

[0030] Fifth, it has a wide range of flexible adaptability. The dual sponge suction cup combination of the end effector can flexibly choose to adsorb individually or simultaneously according to the size of the workpiece, adapting to non-standard workpieces of different sizes and materials without changing the fixture, and adapting to a variety of production scenarios.

[0031] The above overview is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further embodiments and features of this disclosure will be readily apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0032] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments disclosed in this disclosure and should not be construed as limiting the scope of this disclosure.

[0033] Figure 1 This is a diagram of a dynamic scanning, positioning, and grasping system based on J1-axis homing vision in an embodiment of this disclosure;

[0034] Figure 2 This is a detailed enlarged view of the end effector in the embodiments of this disclosure;

[0035] Figure 3 This is a flowchart of the dynamic scanning positioning and grasping method based on J1-axis homing vision in the embodiments of this disclosure;

[0036] Figure 4 This is a flowchart illustrating the deep learning-based recognition process in an embodiment of this disclosure. Detailed Implementation

[0037] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of this disclosure. Therefore, the drawings and description are to be considered exemplary in nature and not restrictive.

[0038] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0039] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.

[0040] This disclosure provides a dynamic scanning, positioning, and grasping system based on J1-axis homing vision, including:

[0041] Robotic arm 1, the robotic arm 1 having a J1 axis 11 and a rotating base 12 that rotates synchronously with the J1 axis;

[0042] End effector 2, which is installed at the end of the robotic arm 1 and is used to grasp a workpiece;

[0043] The follow-up vision component 3 includes a global vision unit 32, which covers the working area of ​​the end effector 2. In a preferred embodiment, it also includes a rigid bracket 31, which is fixed to the rotating base 12 and on which the global vision unit 32 is mounted.

[0044] Control component 4, which is communicatively connected to the robotic arm 1, end effector 2, and global vision unit 32, is configured as follows:

[0045] The J1 axis 11 is driven to rotate, thereby enabling the global vision unit 32 to perform multi-angle scanning of the workpiece stack. Multi-view image fusion is used to eliminate reflection interference and complete workpiece positioning. The height information of the workpiece is calculated using the baseline distance formed by the rotation of the J1 axis 11 and the principle of monocular parallax. While the robotic arm 1 is performing the handling operation, the global vision unit 32 is controlled to synchronously complete the scanning of the remaining workpiece stack and pre-calculate the gripping pose of the next workpiece, thus realizing the parallel operation of handling and visual scanning.

[0046] In one embodiment, the control component 4 employs an industrial control computer, which uses an EtherCAT industrial bus to achieve real-time synchronization of the angle feedback of the J1 axis 11, the triggering of the global vision unit 32, the motion control of the robotic arm 1, and the grasping component of the end effector 2. The system pre-calibrates the hand-eye coordinates using the hand-eye calibration algorithm built into OpenCV, establishing a real-time mapping matrix between the global vision unit coordinate system and the robotic arm base coordinate system to ensure accurate conversion of positioning coordinates.

[0047] In one embodiment, the global vision unit 32 is a wide-angle industrial area array camera. The control component 4 detects the proportion of overexposed pixels and the gradient variance of the image, and selects the image frame with the least reflection and the highest clarity as the positioning image, or stitches and fuses the effective areas without overexposure of multiple frames to generate a workpiece positioning image without reflection interference. The rigid bracket 31 is vertically fixed on the rotating base 12 (rotating synchronously with the J1 axis 11, and can rotate along a certain radius arc according to the installation position of the wide-angle industrial area array camera). The height of the rigid bracket 31 is 800mm to 2000mm (preferred embodiment is 1500mm), and the field of view vertically covers the entire working area with a diameter of 4m. Those skilled in the art can also select a suitable height according to the height of the J1 axis. The global camera is installed at the top of the rigid bracket 31, and the field of view covers the working area of ​​the robotic arm 1. The rotation of the J1 axis 11 drives the top camera to perform a circumferential scan around the workpiece, and the reflection interference is eliminated through multi-view image fusion to achieve "anti-reflective imaging".

[0048] In one embodiment, for applications where the total stacking height of the workpiece is low (lower than the height of the J1 axis base), another simplified embodiment can be adopted, namely, using a horizontal ultra-wide-angle (such as dual-camera image stitching) global vision unit, thereby omitting the rigid support and directly mounting the ultra-wide-angle camera on the rotating base of the J1 axis. The camera is angled downwards to the other side of the robotic arm to take pictures (avoiding obstruction by the robotic arm). At this time, the working area of ​​the robotic arm is also located on the other side, and the working process is similar to the previous embodiment.

[0049] In one embodiment, the control component 4 constructs a monocular stereo vision model based on the rotation angle of the J1 axis 11 and the installation height of the global vision unit 32. By matching workpiece feature points in images from different viewpoints, it calculates the disparity displacement and inversely solves for the height of the top surface of the workpiece stack, achieving Z-axis coordinate measurement without the need for an additional 3D camera. An example of the calculation process for disparity displacement by matching workpiece feature points in images from different viewpoints to inversely solve for the height of the top surface of the workpiece stack is as follows:

[0050] (1) Calculation of baseline distance B: Camera installation height H = 1500mm + 1500mm (base height);

[0051] If the rotation angles of axis J1 are θ1 and θ2 (e.g., -20° and +20°), then the baseline distance B = H × (sinθ2 - sinθ1);

[0052] (2) Obtaining the focal length f: It can be obtained through camera calibration (unit: pixels). During the calibration process, standard patterns such as checkerboard are used to obtain the camera intrinsic parameter matrix;

[0053] (3) Parallax calculation process: Rotate the J1 axis to angle θ1, and the global camera captures image I1. Rotate the J1 axis to angle θ2, and the global camera captures image I2. Extract feature points P1 (such as edge points of the board) in image I1, find the corresponding feature points P2 in image I2, and calculate the parallax d = u1 - u2 (horizontal coordinate difference).

[0054] (4) Depth calculation: Substitute into the formula Calculate the depth, Z, which represents the distance from the feature point to the camera;

[0055] (5) In the scenario of board stacking / destacking, height measurement is achieved in the following ways:

[0056] The initial scan determines the position of the pallet surface (reference plane); the reference plane depth Z0 is obtained through calibration or calculated from a known height; the height of the top plate is calculated by measuring the depth Z of the feature points on the top surface of the plate; then the plate height h = Z0 - Z; multiple feature points can be selected for measurement, and the accuracy can be improved by minimum variance estimation or weighted average.

[0057] In a preferred embodiment, reflective areas are also removed by multi-frame image fusion: multiple frames of images are continuously acquired during the J1 axis 11 scanning process, and reflective areas are removed by an image fusion algorithm, mainly based on the following characteristics: position change: the position of the reflective area will change when the shooting angle or light source changes; intensity difference: the brightness of the reflective area is different under different exposure conditions; physical characteristics: reflective areas usually have high brightness and low texture characteristics; finally, a virtual noise-free image is generated to improve the accuracy of feature point extraction.

[0058] In one embodiment, such as Figure 2 As shown, the system also includes a local depth vision sensor 21 mounted on the end effector 2. The control component 4 first uses the global vision unit 32 to perform large-scale coarse positioning of the workpiece, then guides the end effector 2 to move above the gripping point, and then uses the local depth vision sensor 21 to perform close-range fine positioning. Based on the fine positioning results, the system corrects the data deviation of the coarse positioning and updates the workpiece gripping pose. To maximize production efficiency, the system adopts a parallel operation mechanism of handling and visual scanning, achieving zero-time visual inspection. Specifically, after the system detects the stack of workpieces on the pallet through the global vision unit 32, it generates an initial coarse gripping pose for the top workpiece. The robotic arm 1 moves to above the gripping point based on the coarse gripping pose, and the local depth vision sensor 21 on the end effector 2 refines the workpiece position and height, compressing the positioning error to within ±1mm. Then, the gripping component is controlled to pick up the top workpiece. The robotic arm 1 lifts the workpiece and moves it to the unloading position; this process takes approximately 2 to 5 seconds, and this stage is the parallel operation stage. As robotic arm 1 leaves the picking position, control component 4 drives J1 axis 11 to rotate independently. Global vision unit 32 continuously scans and acquires images of the remaining workpiece stack. The background processes the acquired image sequence in real time, performing anti-glare fusion, workpiece recognition, and height calculation to pre-calculate the gripping pose of the next workpiece. When robotic arm 1 returns to the picking position after completing the unloading action, the pose for the next gripping action has already been calculated. Robotic arm 1 does not need to wait for photography and calculation; it can directly execute the next gripping action, achieving seamless continuous operation.

[0059] The system employs a dual macroscopic and microscopic positioning mechanism to ensure gripping accuracy during long-term continuous operation. The first stage is macroscopic coarse positioning, where a global vision unit 32 on a rigid support 31 performs a large-scale search and positioning, covering the entire work area and quickly determining the approximate position, size, and stacking height of the workpiece, allowing for large-angle tilting and positional shifts. The second stage is microscopic fine positioning. Based on the coarse positioning results, the robotic arm 1 moves to a position 200mm to 500mm above the gripping point, and a local depth vision sensor 21 on the end effector 2 performs close-range imaging to identify the precise height and edge position of the workpiece. The control component 4 compares the fine positioning coordinates collected by the local depth vision sensor 21 with the coarse positioning coordinates output by the global vision unit 32, correcting system data deviations within the error range to offset zero-point drift and accumulated errors caused by long-term operation of the robotic arm 1. If the local depth vision sensor 21 detects workpiece sliding or stacking misalignment, or discovers a higher top-level workpiece profile, it immediately updates the gripping pose of the top-level workpiece and replans the gripping path to avoid gripping failure.

[0060] like Figure 2As shown, the end effector 2 also includes a gripping component 22, which is a combination of two sponge suction cups, including a wide suction cup and a narrow suction cup, capable of adsorbing workpieces individually or simultaneously depending on their size. It can adapt to non-standard workpieces ranging from 50mm to 2440mm.

[0061] This disclosure also discloses a dynamic scanning positioning and grasping method based on J1-axis servo vision, such as... Figure 3 As shown, it includes the following steps:

[0062] S10. Fix the rigid bracket 31 to the base rotation 12 of the robotic arm 1 so that the global vision unit 32 installed on the top of the rigid bracket 31 can rotate synchronously with the J1 axis 11.

[0063] S20 drives the J1 axis 11 to reciprocate within a preset angle range, driving the global vision unit 32 to acquire multi-angle images of the workpiece stack, and removes reflection interference and identifies the workpiece outline through multi-view image fusion.

[0064] In one embodiment, a single-stage object detection algorithm based on deep learning (YOLO series, such as YOLOv8 or YOLOv5) can be used as the core recognition engine. This algorithm uses an end-to-end convolutional neural network to directly regress the category, bounding box coordinates, and confidence score of the top-level object from the acquired image sequence. Figure 4 As shown, the recognition process is as follows:

[0065] S201. Image preprocessing and enhancement: Receive the RGB image captured by the camera, scale it to the model input size, and use the Mosaic data augmentation strategy to stitch together multiple workpiece images from different perspectives to enrich the background diversity and improve the model's robustness to workpiece edges and occlusion.

[0066] S203, Feature Extraction (Backbone): Utilizes a backbone network (such as CSPDarknet) to perform multi-level feature extraction on the input image. Introducing CBAM (Convolutional Block Attention Module) or SE (Squeeze-and-Excitation) attention mechanisms enables the network to automatically suppress background noise (such as tray texture and reflective points), focusing on the edge contours and corner features of the board material.

[0067] S205, Feature Fusion (Neck), uses a Feature Pyramid Network (FPN) and Path Aggregation Network (PANet) structure to fuse deep semantic features with shallow geometric features. For detecting workpieces of different sizes, such as sheet metal (small and large pieces), it ensures accurate localization regardless of the sheet metal's proportion in the frame from the camera's top-down view.

[0068] S207. Head Output and Pose Calculation: The head outputs the workpiece's category probability and bounding box coordinates. In addition to the standard bounding box, the algorithm also regresses the coordinates of the four corner points or the minimum bounding rectangle of the workpiece. Combining the rotation angle of the J1 axis and the camera intrinsic parameters, the PnP (Perspective-n-Point) algorithm is used to calculate the 6-DOF pose of the workpiece in the robot arm's base coordinate system.

[0069] S30, Based on the baseline distance formed by rotating J1 axis 11, the height information of the top surface of the workpiece stack is calculated by the monocular parallax principle;

[0070] In one embodiment, the control component 4 constructs a monocular stereo vision model based on the rotation angle of the J1 axis 11 and the installation height of the global vision unit 32. By matching workpiece feature points in images from different viewpoints, it calculates the disparity displacement and inversely solves for the height of the top surface of the workpiece stack, achieving Z-axis coordinate measurement without the need for an additional 3D camera. An example of the calculation process for disparity displacement by matching workpiece feature points in images from different viewpoints to inversely solve for the height of the top surface of the workpiece stack is as follows:

[0071] (1) Calculation of baseline distance B: Camera installation height H = 1500mm + 1500mm (base height);

[0072] If the rotation angles of axis J1 are θ1 and θ2 (e.g., -20° and +20°), then the baseline distance B = H × (sinθ2 - sinθ1);

[0073] (6) Obtaining the focal length f: It can be obtained through camera calibration (unit: pixels). During the calibration process, standard patterns such as checkerboard are used to obtain the camera intrinsic parameter matrix;

[0074] (7) Parallax calculation process: Rotate the J1 axis to angle θ1, and the global camera captures image I1. Rotate the J1 axis to angle θ2, and the global camera captures image I2. Extract feature points P1 (such as edge points of the board) in image I1, find the corresponding feature points P2 in image I2, and calculate the parallax d = u1 - u2 (horizontal coordinate difference).

[0075] (8) Depth calculation: Substitute into the formula Calculate the depth, Z, which represents the distance from the feature point to the camera;

[0076] (9) In the scenario of board stacking / destacking, height measurement is achieved in the following ways:

[0077] The initial scan determines the position of the pallet surface (reference plane); the reference plane depth Z0 is obtained through calibration or calculated from a known height; the height of the top plate is calculated by measuring the depth Z of the feature points on the top surface of the plate; then the plate height h = Z0 - Z; multiple feature points can be selected for measurement, and the accuracy can be improved by minimum variance estimation or weighted average.

[0078] S40. During the workpiece handling operation performed by the robotic arm, the J1 axis 11 and the global vision unit 32 are synchronously driven to complete the scanning of the remaining workpiece stack and pre-calculate the gripping pose of the next workpiece.

[0079] After S50 and robotic arm 11 complete the unloading and return to the picking position, they directly execute the next picking based on the pre-calculated picking posture, realizing continuous operation without waiting.

[0080] In one embodiment, the step of removing reflective interference through multi-view image fusion specifically includes: calculating the gradient variance and overexposed pixel ratio of each frame in real time, selecting the image frame with the least reflective interference and the highest clarity as the positioning image, or performing pixel-level stitching and fusion of the non-overexposed areas of multiple frames to generate a virtual positioning image without reflective interference.

[0081] In one embodiment, the step of calculating height information using the monocular parallax principle specifically includes: controlling the J1 axis to rotate to two different angles to form a measurement baseline, extracting the parallax displacement of the same workpiece feature point in images from different viewpoints, and combining the installation height of the global vision unit with the rotation angle of the J1 axis to solve for the top surface height of the workpiece stack.

[0082] In one embodiment, a dual positioning step is also included: first, the global vision unit 32 completes the large-scale coarse positioning of the workpiece; after the robotic arm 11 moves above the gripping point, the local depth vision sensor 21 integrated in the end effector 2 completes the close-range fine positioning; and the data deviation of the coarse positioning and the gripping pose are corrected according to the fine positioning result.

[0083] In one embodiment, in the step of driving the J1 axis 11 to reciprocate within a preset angle range, the preset angle range is ±30°, and the time taken for the robotic arm 11 to perform the plate handling operation is 2 to 5 seconds. During this time, the scanning of the remaining plate stack and the pre-calculation of the next pose are completed.

[0084] The above are merely specific embodiments of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in this disclosure, and these should all be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A dynamic scanning positioning and grasping system based on J1-axis servo vision, characterized in that, Comprise: A mechanical arm with a J1 axis and a rotating base that rotates synchronously with the J1 axis; An end effector mounted at the end of the mechanical arm for grabbing workpieces; A servo vision assembly including a global vision unit whose field of view covers the working area of the end effector; A control assembly in communication with the mechanical arm, end effector and global vision unit; configured to: Drive the J1 axis to rotate to drive the global vision unit to perform multi-angle scanning on the workpiece stack, eliminate glare interference through multi-view image fusion and complete workpiece positioning; use the baseline distance formed by the rotation of the J1 axis to calculate the height information of the workpiece according to the monocular parallax principle; while the mechanical arm performs the carrying operation, control the global vision unit to complete the scanning of the remaining workpiece stack synchronously, and pre-calculate the grabbing pose of the next workpiece, realizing parallel operation of carrying and vision scanning.

2. The Jl-axis-following-vision-based dynamic scanning positioning and grasping system of claim 1, wherein, The global vision unit is a wide-angle industrial area array camera, and the control assembly selects the image frame with the least glare and the highest clarity as the positioning image by detecting the overexposed pixel ratio and gradient variance of the image, or splices and fuses the non-overexposed effective areas of multiple image frames to generate a workpiece positioning image without glare interference.

3. The dynamic scanning positioning and grasping system based on Jl-axis servo vision of claim 1 or 2, characterized in that, The control assembly constructs a monocular stereo vision model based on the rotation angle of the J1 axis and the installation height of the global vision unit, calculates the parallax displacement by matching the workpiece feature points in different view images, and inversely solves the height of the top surface of the workpiece stack, realizing Z-axis coordinate measurement without additional three-dimensional cameras.

4. The dynamic scanning positioning and grasping system based on Jl-axis servo vision of claim 1 or 2, wherein, It also includes a local depth vision sensor arranged on the end effector, wherein the control assembly first uses the global vision unit to complete the rough positioning of the workpiece in a large range, then guides the end effector to move above the grabbing point, then uses the local depth vision sensor to complete the close-range fine positioning, and corrects the data deviation of the rough positioning according to the fine positioning result to update the grabbing pose of the workpiece.

5. The Jl-axis-following-vision-based dynamic scanning positioning and grasping system according to claim 1 or 2, characterized in that, The end effector also includes a grabbing assembly, which is a double-sponge suction disc combination including a wide suction disc and a narrow suction disc, and can individually or simultaneously adsorb according to the size of the workpiece.

6. The dynamic scanning positioning and grabbing method based on J1 axis servo vision, characterized in that, Comprise the following steps: Fix a rigid support on the rotating base of the mechanical arm, so that the global vision unit mounted at the top end of the rigid support can rotate synchronously with the J1 axis; drive the J1 axis to reciprocate within a predetermined angle range to drive the global vision unit to collect multi-angle images of the workpiece stack, remove glare interference and identify the workpiece contour through multi-view image fusion; calculate the height information of the top surface of the workpiece stack based on the baseline distance formed by the rotation of the J1 axis; during the workpiece carrying operation of the mechanical arm, drive the J1 axis and the global vision unit to complete the scanning of the remaining workpiece stack synchronously, and pre-calculate the grabbing pose of the next workpiece; after the mechanical arm completes the unloading and returns to the picking position, directly executes the next grabbing according to the pre-calculated grabbing pose, realizing continuous operation without waiting.

7. The dynamic scanning positioning and grasping method based on Jl-axis servo vision of claim 6, wherein, The step of removing reflective interference through multi-view image fusion specifically includes: calculating the gradient variance and overexposed pixel ratio of each frame in real time, selecting the image frame with the least reflective interference and the highest clarity as the positioning image, or performing pixel-level stitching and fusion of the non-overexposed areas of multiple frames to generate a virtual positioning image without reflective interference.

8. The dynamic scanning positioning and grasping method based on Jl-axis servo vision of claim 6, wherein, The steps for calculating height information using the monocular parallax principle specifically include: controlling the J1 axis to rotate to two different angles to form a measurement baseline, extracting the parallax displacement of the same workpiece feature point in images from different viewpoints, and combining the installation height of the global vision unit with the rotation angle of the J1 axis to solve for the top surface height of the workpiece stack.

9. The dynamic scanning positioning and grasping method based on Jl-axis servo vision of claim 6, wherein, It also includes a dual positioning process: first, the workpiece is coarsely positioned over a large area using a global vision unit; then, after the robotic arm moves above the gripping point, fine positioning is performed at close range using a local depth vision sensor integrated into the end effector; and the data deviation of the coarse positioning and the gripping pose are corrected based on the fine positioning results.

10. The dynamic scanning positioning and grasping method based on Jl-axis servo vision of claim 6, wherein, In the step of driving the J1 axis to reciprocate within a preset angle range of ±30°, the time taken for the robotic arm to perform the plate handling operation is 2 to 5 seconds. During this time, the scanning of the remaining plate stack and the pre-calculation of the next pose are completed.

Images