Nickel strip automatic arrangement method and system based on visual guidance

Abstract

The present application relates to the technical field of industrial robots, and discloses a nickel strip automatic arrangement method based on visual guidance, which comprises the following steps: S1: taking the right boundary of a conveying belt as a positioning reference, collecting an image of the conveying belt, extracting edge points, and then extracting the right boundary straight line of the conveying belt by combining a Hough straight line transformation; collecting an image of a first nickel strip, extracting edge points, and then extracting the right boundary straight line of the first nickel strip by combining a Hough straight line transformation. The present application collects images of the conveying belt and the nickel strip in real time by using an industrial camera, extracts the boundary straight line of the conveying belt and the boundary straight line of the nickel strip, adjusts the end pose of a robot by using a visual closed-loop control mode, makes the included angle and the perpendicular distance between the right boundary straight line of the nickel strip and the boundary straight line of the positioning reference meet preset conditions respectively, realizes accurate control of the placement position and the attitude of the nickel strip, and avoids positioning deviation caused by physical contact and size fluctuation in a traditional mechanical positioning mode.

Description

Technical Field

[0001] This invention relates to the field of industrial robot technology, and in particular to a vision-guided automatic nickel bar arrangement method and system. Background Technology

[0002] In the manufacturing process of lithium batteries and other products, nickel bars (or nickel sheets, nickel-plated tabs) are important conductive connecting components. They need to be neatly arranged in rows on a conveyor belt at set intervals for automated handling and assembly in subsequent processes. Currently, the arrangement of nickel bars mainly relies on manual placement or automated equipment based on mechanical positioning. Manual placement suffers from low efficiency, poor accuracy, and high labor intensity; while existing automated placement devices based on mechanical positioning typically rely on physical blocks, limiting channels, or slides on the conveyor belt to directionally transport and laterally arrange the materials.

[0003] However, the aforementioned existing technologies have the following drawbacks: Firstly, mechanical positioning relies on the physical contact between the material and the guiding mechanism. When the nickel bar surface is uneven or has slight dimensional fluctuations, jamming or positioning deviations can easily occur. Secondly, during the process of arranging nickel bars one by one, if the placement position of each nickel bar is based on the edge of the conveyor belt, the error in the arrangement spacing will accumulate with the increase in the number of nickel bars, making it difficult to guarantee the overall straightness and spacing uniformity of the entire row of nickel bars. In addition, the existing solutions lack the ability to detect and correct the nickel bar gripping posture (such as angular deviation) in real time. When the robot grips the nickel bars at an angular deviation, the placed nickel bars cannot maintain a strict parallel relationship, further affecting the arrangement quality. These drawbacks make it difficult for the existing nickel bar arrangement methods to meet the high requirements of lithium battery production lines for arrangement accuracy and consistency. Summary of the Invention

[0004] To address the technical problems mentioned in the background section, this invention provides a visually guided automatic arrangement method for nickel bars.

[0005] This invention is achieved using the following technical solution: a vision-guided automatic nickel bar arrangement method, comprising the following steps:

[0006] S1: Using the right boundary of the conveyor belt as the positioning reference, acquire an image of the conveyor belt, extract edge points, and then combine Hough line transform to extract the right boundary line of the conveyor belt. Acquire an image of the first nickel bar, extract edge points, and then combine Hough line transform to extract the right boundary line of the first nickel bar. Adjust the pose of the first nickel bar so that the angle between its right boundary line and the right boundary line of the conveyor belt is less than or equal to a preset angle threshold, and the vertical distance between them is equal to a preset offset reference distance. Place the first nickel bar on the conveyor belt.

[0007] S2: Starting from the second nickel bar, using the left boundary of the most recently placed nickel bar as the positioning reference, acquire the image of the (N-1)th nickel bar that has been placed. After extracting the edge points, combine the Hough line transform to extract the left boundary line of the (N-1)th nickel bar that has been placed. Acquire the image of the currently captured Nth nickel bar. After extracting the edge points, combine the Hough line transform to extract the right boundary line of the Nth nickel bar. Adjust the pose of the Nth nickel bar so that the angle between its right boundary line and the left boundary line of the (N-1)th nickel bar that has been placed is less than or equal to a preset angle threshold, and the vertical distance between them is equal to a preset offset distance. Place the Nth nickel bar on the conveyor belt. N is an integer greater than or equal to 2.

[0008] S3: Establish a reference center line with the center point of the first nickel bar as the reference, extract the coordinates of the center point of the currently placed Nth nickel bar, calculate the axial misalignment deviation of the center point coordinates relative to the reference center line, and apply a fine adjustment amount opposite to the direction of the axial misalignment deviation along the long side of the boundary of the Nth nickel bar on the basis of satisfying the pose adjustment in S2, so that the center point of the Nth nickel bar falls on the reference center line.

[0009] S4: Repeat steps S2 to S3 to place subsequent nickel bars in sequence;

[0010] S5: After each placement is completed, an image of the conveyor belt is acquired, edge points are extracted, and the left boundary line of the conveyor belt is extracted using Hough transform. The vertical distance between the left boundary line of the most recently placed nickel bar and the left boundary line of the conveyor belt is calculated as the remaining distance. When the remaining distance is less than a preset termination threshold, or when the difference between the current remaining distance and the preset offset distance is used as the estimated remaining distance after the next nickel bar is placed, the placement process is stopped.

[0011] Furthermore, before executing S1, hand-eye calibration of the industrial camera is performed to establish the conversion relationship between image pixel coordinates and robot physical coordinates, and the coordinates of the boundary lines and center points identified in the image space are converted to the robot physical coordinate system.

[0012] Furthermore, the preset offset reference distance, the preset offset distance, and the preset termination threshold are set through a human-machine interface; the preset offset distance is less than the preset offset reference distance; and the preset angle threshold is 0.5°.

[0013] Furthermore, after acquiring images in S1 and S2 but before extracting edge points, preprocessing of the acquired images is also included:

[0014] Scale the original image proportionally;

[0015] Extract the region of interest containing the conveyor belt boundary and the nickel bar target from the scaled image;

[0016] Bilateral filtering is applied to the region of interest for noise reduction.

[0017] Convert the filtered image to a grayscale image;

[0018] Perform binarization segmentation on the grayscale image to obtain a binary image;

[0019] Morphological processing is performed on the binary image to remove noise points.

[0020] Furthermore, performing binarization segmentation on the grayscale image to obtain a binary image includes:

[0021] Perform fixed threshold segmentation on the grayscale image to obtain a preliminary binary image;

[0022] The preliminary binary image is inverted so that the target area of ​​the nickel bar is white and the background is black;

[0023] Using the inverted binary image as a mask, the grayscale value of the corresponding nickel bar target area in the grayscale image is extracted;

[0024] The gray values ​​of the extracted nickel bar target area were binarized twice using the Otsu automatic thresholding method to obtain the final binary image.

[0025] Furthermore, the morphological processing also includes:

[0026] Extract all external contours from the binary image;

[0027] Calculate the area and perimeter of each contour, and remove contours with an area smaller than a preset area threshold or a perimeter smaller than a preset perimeter threshold.

[0028] The retained effective contour is fitted using the minimum bounding rectangle of rotation to obtain the boundary vertex coordinates, center point coordinates, size information, and rotation angle of the nickel bar.

[0029] Furthermore, the establishment of a reference center line based on the center point of the first nickel bar, as described in S3, includes:

[0030] Extract the coordinates of the center point of the first nickel bar;

[0031] The reference center line is established using the coordinates of the center point as the reference point and the direction parallel to the direction of the conveyor belt movement as the reference direction.

[0032] Furthermore, an interference check step is included before each nickel bar is placed:

[0033] Based on the current position and orientation of the nickel bars already placed, the boundary position of the conveyor belt, and the robot's current trajectory, determine whether the robot will collide with the already placed nickel bars or the edge of the conveyor belt when it performs the next placement action.

[0034] If there is a risk of collision, adjust the robot's movement path or issue a pause command.

[0035] This invention also proposes a vision-guided automatic nickel bar arrangement system, comprising:

[0036] An industrial robot with an end effector equipped with grippers for grasping and releasing nickel bars;

[0037] Conveyor belts are used to carry the placed nickel bars and provide an arrangement working surface;

[0038] An industrial camera, mounted above the conveyor belt, is used to capture image information of the conveyor belt and the nickel bars;

[0039] The control module is connected to the industrial robot, the fixture, the conveyor belt and the industrial camera respectively. The control module integrates an image processing unit and a motion planning unit, and is used to execute the vision-guided automatic nickel bar arrangement method according to any one of claims 1 to 8.

[0040] Furthermore, the clamp is a magnetic clamp, and a proximity sensor is installed on the magnetic clamp to detect whether the nickel bar is flat and adsorbed.

[0041] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0042] This invention uses an industrial camera to capture images of the conveyor belt and nickel bars in real time. It combines edge detection and Hough line transform to extract the boundary lines of the conveyor belt and the nickel bars. Then, it uses a visual closed-loop control method to adjust the robot's end-effector pose so that the angle and vertical distance between the right boundary line of the nickel bar and the positioning reference boundary line meet preset conditions. This achieves precise control over the placement position and posture of the nickel bars, avoiding the positioning deviations caused by physical contact and size fluctuations in traditional mechanical positioning methods, and significantly improving the placement accuracy of a single nickel bar.

[0043] This invention uses the right edge of the conveyor belt as the positioning reference when placing the first nickel bar, and the left edge of the most recently placed nickel bar as the positioning reference when placing the second nickel bar. It adopts a step-by-step positioning method, so that the position of each newly placed nickel bar depends only on the actual position of the previous placed nickel bar, rather than on ideal coordinates or the fixed reference of the conveyor belt. This effectively avoids the problem of the accumulation of spacing error as the number of nickel bars increases, and ensures the consistency of the spacing of the entire row of nickel bars.

[0044] This invention establishes a reference centerline based on the center point of the first nickel bar, and applies a fine adjustment along the long side of the nickel bar boundary that is opposite to the axial misalignment deviation, while meeting the boundary spacing conditions. This forces the center point of each newly placed nickel bar back to the reference centerline, achieving a straight alignment of the centerlines of the entire row of nickel bars. At the same time, since the fine adjustment direction is orthogonal to the boundary spacing direction, this calibration operation does not change the vertical spacing between adjacent nickel bars, thus balancing the requirements for spacing accuracy and straightness.

[0045] This invention identifies the left boundary of the conveyor belt in real time and calculates the remaining distance between the left boundary of the placed nickel bar and the left boundary of the conveyor belt. It then combines the current remaining distance with a preset offset distance to estimate the remaining space after the next nickel bar is placed. When the remaining space is insufficient, the placement process is automatically stopped, thus realizing the automated cycle and intelligent termination of the nickel bar arrangement operation. The arrangement of the entire row of nickel bars can be completed without manual intervention, improving production efficiency and automation level. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of the hardware structure of the vision-guided automatic nickel bar arrangement system of the present invention; the diagram shows the industrial robot and its end effector-mounted magnetic clamp, conveyor belt, industrial camera and control module.

[0047] Figure 2 This is a schematic diagram illustrating the placement and positioning principle of the first nickel bar in this invention; the straight line L on the right boundary of the conveyor belt is marked in the diagram. belt_R The straight line L on the right boundary of the first nickel bar Ni1_R , Preset offset reference distance d0.

[0048] Figure 3 This is a schematic diagram illustrating the recursive positioning principle of the nickel bars in this invention; the diagram marks the left boundary line L of the (N-1)th nickel bar that has been placed. Ni(N-1)_L The right boundary line L of the Nth nickel bar currently being grabbed Ni(N)_R , Preset offset distance d.

[0049] Figure 4 This is a schematic diagram illustrating the termination condition determination of the present invention; the diagram shows the left boundary line of the conveyor belt, the left boundary line of the nearest placed nickel bar, and the remaining distance D. left It also indicates the preset termination interval.

[0050] Figure 5 This is the overall control flowchart of the present invention; the diagram shows the complete control logic from system startup to process end, and is used to demonstrate the overall execution flow and loop judgment mechanism of the method of the present invention.

[0051] Figure 6This is an overall flowchart of the visual recognition software module of the present invention; the diagram shows the complete software processing flow from the acquisition of raw images by an industrial camera, through a series of processing steps, to the final output of the recognition result.

[0052] Figure 7 This is a schematic diagram of the region of interest (ROI) and binarization process of the nickel bar in this invention; the left side is the ROI image containing the nickel bar target, which is cropped from the scaled image, and the right side is the binary image of the nickel bar target after composite binarization segmentation.

[0053] Figure 8 This is a schematic diagram of nickel bar contour extraction and minimum bounding rectangle fitting according to the present invention. The diagram shows the effective nickel bar contours retained after morphological processing and contour screening, as well as the rectangular envelope obtained by fitting the effective contours with the minimum bounding rectangle. Detailed Implementation

[0054] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0055] Example 1

[0056] like Figure 1 As shown, the vision-guided automatic nickel bar arranging system used in this invention mainly includes: an industrial robot, a magnetic gripper, a conveyor belt, an industrial camera, and a control module. The end effector of the industrial robot is equipped with a magnetic gripper for picking up and releasing nickel bars. The conveyor belt carries the placed nickel bars and provides the arranging work surface. The industrial camera is mounted above the conveyor belt, its field of view covering the nickel bar placement area, for real-time acquisition of image information of the conveyor belt and nickel bars. The control module is electrically connected to the industrial robot, the magnetic gripper, the conveyor belt, and the industrial camera, and integrates an image processing unit and a motion planning unit to receive visual recognition results and generate robot motion control commands.

[0057] In this embodiment, the industrial camera is a 12-megapixel global shutter camera with a lens focal length of 8mm, an installation height of 300mm, and a field of view of approximately 200mm × 150mm. It should be noted that the above parameters can be adjusted according to actual working conditions and accuracy requirements, and this invention does not impose specific limitations on them.

[0058] It's worth mentioning that the magnetic gripper is also equipped with a proximity sensor to detect whether the nickel bar is properly and evenly adsorbed, preventing collisions caused by misaligned adsorption. The detection signal from this proximity sensor is read by the control module before each placement action by the robot. Subsequent placement actions are only allowed when the nickel bar is detected to be properly and evenly adsorbed; if misalignment is detected, the control module issues an alarm signal and suspends the operation.

[0059] In this embodiment, the parameter preset and system calibration are implemented as follows:

[0060] Before performing the nickel bar arrangement operation, the parameters are preset and the system is calibrated.

[0061] The operator sets the initial offset reference distance d0 through the human-machine interface. In this embodiment, d0 is set to 50mm. The preset offset distance d is set to 20mm, and can be adjusted within the range of 10mm to 50mm through the human-machine interface. In addition, a termination distance range also needs to be set; in this embodiment, the preset termination range is 20mm to 50mm.

[0062] To achieve a precise mapping between visual recognition results and robot physical coordinates, hand-eye calibration of the vision system is required. Specifically, this involves establishing a mapping between camera pixel coordinates (u, v) and robot physical coordinates (u, v). , The homogeneous transformation relationship between ) is shown in Equation (1).

[0063] (1)

[0064] The coefficient a in formula (1) 11 To a 23 The transformation matrix parameters obtained from hand-eye calibration are used to convert the visual information loop, such as boundary coordinates and center point coordinates detected in the image space, into physical coordinates in the robot's execution space, thereby ensuring that the robot can accurately locate itself based on the visual recognition results. After calibration, the pixel equivalent in this embodiment is 0.05 mm / pixel.

[0065] Where u and v represent the horizontal and vertical pixel coordinates in the camera image pixel coordinate system, respectively; , These represent the horizontal and vertical physical coordinates in the robot's physical coordinate system, respectively; the constant 1 in the matrix represents the homogeneous coordinate term, used to convert the two-dimensional pixel coordinates into homogeneous coordinate form.

[0066] Furthermore, this embodiment predefines the identification features of each boundary: the left and right boundaries of the conveyor belt are straight lines with edges, and the boundary of the nickel bar is a straight line along the long side of the rectangular nickel bar. The control module also sets the robot's movement speed according to production cycle requirements—50mm / s for low-speed placement and 200mm / s for high-speed idle—to balance placement accuracy and operational efficiency. During edge detection, an edge detection threshold and a minimum of 20 edge points are set for straight line fitting.

[0067] In this embodiment, the first nickel bar is placed with the right edge of the conveyor belt as the positioning reference. For example... Figure 2 As shown, the specific process is as follows:

[0068] An industrial camera captures real-time images of the conveyor belt placement area. The image processing unit performs visual recognition processing on the captured images (the specific image processing flow will be detailed later). Edge points in the image are extracted using the Canny edge detection algorithm, and then the right boundary line of the conveyor belt is extracted using the Hough line transform, denoted as... .

[0069] Simultaneously, the robot uses a magnetic gripper to pick up the first nickel bar from the feed tray and moves it into the field of view of the vision system. The vision system identifies the right boundary line of the nickel bar—that is, the long side of the nickel bar closest to the right side of the conveyor belt—and records it as... .

[0070] The control module calculates the target placement pose. For example... Figure 2 As shown, the target pose needs to satisfy the following two conditions simultaneously:

[0071] Parallelism condition: Straight line of the right boundary of the conveyor belt Straight line with the right boundary of the first nickel bar The included angle θ between the two lines should be less than or equal to the preset threshold. The specific calculation formula for the included angle between the two lines is shown in formula (2).

[0072] (2)

[0073] Where θ represents the angle between the two boundary lines to be compared; it should be noted that line 1 and line 2 are respectively represented as... and A1, B1 and A2, B2 are the coefficients of the x and y terms in the two linear equations, respectively, and C1 and C2 are the constant terms in the corresponding linear equations. When formula (2) is used to place the first nickel bar, line 1 is the right boundary line of the conveyor belt, and line 2 is the right boundary line of the first nickel bar; when formula (2) is used to place subsequent nickel bars, line 1 is the left boundary line of the (N-1)th nickel bar that has been placed, and line 2 is the right boundary line of the Nth nickel bar.

[0074] In this embodiment, the angle threshold is set to 0.5°, which means that the angle between the right edge of the nickel bar and the right edge of the conveyor belt is not more than 0.5°, so as to ensure that the first nickel bar maintains a strict parallel relationship with the edge of the conveyor belt.

[0075] Offset condition: Right boundary of the conveyor belt Right boundary of the first nickel bar The horizontal distance between them (measured along the direction perpendicular to the boundary) is exactly equal to the preset offset reference distance d0. The formula for calculating this vertical distance is shown in formula (3), and it requires that the nickel strip is located inside the boundary of the conveyor belt. In this embodiment,

[0076] That is, the first nickel bar is placed 50mm inside the right boundary of the conveyor belt.

[0077] (3)

[0078] Where d represents the vertical distance between the two boundary lines measured along their normal direction; C1 and C2 represent the constant terms in the equations of the two boundary lines to be calculated, respectively; A1 and B1 represent the coefficients of the x and y terms in the equation of the reference line used for distance calculation. Before performing distance calculation, both boundary lines are transformed to the same robot physical coordinate system and expressed in the same normal direction.

[0079] In the specific calculation process, if the visual recognition detects that the currently grasped nickel bar has an angular deviation—for example, in this embodiment, the initial grasping angle has a deviation of 1.2°—the control module calculates the required rotation correction amount (rotation -1.2°) and translation amount, and generates a complete robot end-effector pose command.

[0080] The robot places the first nickel bar onto the conveyor belt based on its calculated pose and slightly releases the magnetic force to confirm stable placement. After placement, the vision system simultaneously records the coordinates of the center point C1 of the first nickel bar, which will serve as the basis for establishing the reference center line later.

[0081] In this embodiment, starting from the second nickel bar, the placement of each subsequent nickel bar is based on the left boundary of the previously placed nickel bar, rather than continuing to use the right boundary of the conveyor belt as the reference. For example... Figure 3 As shown, the specific process is as follows:

[0082] The vision system identifies the left boundary line of the (N-1)th nickel bar that has been placed, and denotes it as... Simultaneously, the vision system identifies the right boundary line of the Nth nickel bar currently being grasped by the robot, denoted as... .

[0083] The control module calculates the target placement pose. This pose must meet the following conditions:

[0084] Parallel condition: Left boundary of the (N-1)th nickel bar Right boundary of the Nth nickel bar The included angle between them is less than or equal to a preset threshold (also 0.5° in this embodiment) to ensure that adjacent nickel bars remain strictly parallel. The included angle can be calculated using formula (2).

[0085] Offset condition: Left boundary of the (N-1)th nickel bar Right boundary of the Nth nickel bar The vertical distance between them is exactly equal to the preset offset distance d, and the direction is from the left boundary of the (N-1)th nickel bar to the outside of the right boundary of the Nth nickel bar—that is, all nickel bars are placed in the same direction, arranged sequentially from right to left. The calculation method for this vertical distance can be referred to formula (3).

[0086] In this embodiment, when placing the second nickel bar, the left boundary coordinates of the first nickel bar are visually extracted (e.g., in the robot's base coordinate system). (0° angle), and simultaneously extract the right boundary of the currently grasped second nickel bar, assuming a 0.8° skew. The control module calculates the target pose as: rotation -0.8° to correct the angle deviation, translation to This is the location (i.e., offset 20mm to the left from the left boundary of the first block). This process uses visual closed-loop control to ensure that the errors of each spacing segment are independent and do not accumulate.

[0087] To further ensure the overall straightness of the entire row of nickel bars, this invention adds a real-time collinearity calibration mechanism for center points to the boundary offset control. Specifically, the vision system simultaneously extracts the center point of the Nth nickel bar. The coordinates. Control module requirements. With the established baseline centerline Collinear. The reference center line is a straight line established with the center point C1 of the first nickel bar as the reference and parallel to the direction of the conveyor belt movement.

[0088] During the calibration process, the control module first calculates the axial misalignment deviation δ from the current center point to the reference center line. The specific calculation method is shown in formula (4).

[0089] (4)

[0090] in, This indicates the signed axial misalignment deviation of the center point of the Nth nickel bar relative to the reference center line. , These represent the center points of the Nth nickel bar. The horizontal and vertical coordinates in the robot's physical coordinate system; , , As the reference center line Normalized linear equation The coefficient, and satisfying . The sign is used to indicate the direction of the offset of the center point relative to the reference center line.

[0091] like If the value is not zero—for example, in this embodiment, the current center Y-coordinate of the second nickel bar is +1.3mm, deviating from the baseline—then this deviation needs to be corrected. Based on the initial placement pose calculated to meet the boundary parallel offset condition, the control module applies a physical fine-tuning amount along the straight line of the long side of the nickel bar boundary. This means controlling the robot's end effector to slide along the long side of the nickel bar to compensate for the deviation. Since this fine-tuning direction is orthogonal to the boundary spacing direction, it is possible to eliminate the center point deviation without changing the vertical spacing between adjacent nickel bars. In this embodiment, the current center Y-coordinate of the second nickel bar is +1.3mm. Then apply Fine adjustments were made along the long side to ensure the adjusted center point fell precisely on the reference center line. The final placement of the second nickel bar was x=92mm, y=0mm, and angle 0°.

[0092] In this way, each newly placed nickel bar is forcibly pulled to the reference center line while meeting the boundary spacing accuracy, thereby ensuring that the center line of the entire row of nickel bars is straight.

[0093] It should be noted that the loop execution and termination condition judgment in this embodiment are implemented as follows:

[0094] After placing the second nickel bar, the system sequentially places the third, fourth, ... Nth nickel bars according to the method described in Example 4. After placing each nickel bar (or before placing the next one), the system performs a termination condition check, the specific process of which is as follows:

[0095] The vision system identifies the straight line at the left boundary of the conveyor belt in real time and records it as... .

[0096] Calculate the left boundary of the most recently placed nickel bar. left edge of the conveyor belt The vertical distance between them is denoted as The formula for calculating the vertical distance is shown in formula (5).

[0097] Control module according to The value of the argument terminates the logical judgment. For example... Figure 4 As shown, the preset termination range in this embodiment is 20mm to 50mm. The specific judgment logic is as follows:

[0098] (5)

[0099] in, This represents the remaining distance between the left boundary of the most recently placed nickel bar and the left boundary of the conveyor belt; the straight line representing the left boundary of the most recently placed nickel bar is... The straight line representing the left boundary of the conveyor belt is as follows: ; , These are the constant terms in the equations of the two parallel boundary lines mentioned above; , The common normal coefficients used for calculating the remaining distance. During the calculation, both lines are located in the same robot physical coordinate system, and their normal directions are unified.

[0100] If 20mm≤ If the value is ≤50mm, the system will continue to execute the next placement cycle.

[0101] like <20mm, or estimate the next nickel bar after placement If the thickness is less than 20mm, the placement process should be stopped immediately. After the next piece is placed... The estimated value is based on the current The calculation is obtained by subtracting the preset offset distance d. That is, for each nickel bar placed, the leftmost boundary of the nickel bar queue will be extended to the left by an offset distance d. If the estimated result is less than 20mm, no more bars will be placed.

[0102] like >50mm, but the remaining space is insufficient to safely place the next nickel bar (i.e., after placement). If the size jumps to less than 20mm, then the process ends after placing the last piece.

[0103] when When the device enters the 20-50mm range, the system will issue a prompt signal. Optionally, the device can place the last nickel bar that meets the space requirements and then the process will end.

[0104] In this specific application, the system places nickel bars sequentially, and after each bar is placed, the distance between the left boundary of the current nickel bar and the left boundary of the conveyor belt is measured. After placing the 10th nickel bar, the left boundary of the nickel bar was measured to be x=-120mm in the robot's base coordinate system, and the left boundary of the conveyor belt was x=-150mm, with D_left=30mm, falling within the 20-50mm range. The system then checked the offset space (20mm) required for placing the next nickel bar and found that if the 11th bar were placed, its left boundary would reach x=-140mm, with a distance of only 10mm from the left boundary of the conveyor belt, less than the 20mm safety threshold. Therefore, the system did not place the 11th bar and ended the process. In the end, a total of 10 nickel bars were placed in this embodiment, with all adjacent nickel bars spaced 20mm apart, a collinearity deviation of less than 0.3mm, and a distance of 30mm between the end and the left boundary of the conveyor belt.

[0105] It is worth mentioning that before each placement, the vision system also performs an interference check. Specifically, based on the position and orientation of the currently placed nickel bar, the actual position of the conveyor belt boundary, and the robot's current movement trajectory, it determines whether the robot will collide with the already placed nickel bar or the edge of the conveyor belt when performing the next placement action. If there is a risk of collision, the control module adjusts the robot's movement path or issues a pause command. The aforementioned proximity sensor detection and interference check together constitute the multiple safety strategies of this invention.

[0106] Example 2

[0107] In this embodiment, the visual recognition functions involved in Embodiment 1 above are all specifically executed by the OpenCV-based visual recognition software module built into the control module. The following is in conjunction with... Figures 6 to 8 The complete processing flow of this visual recognition software module is described in detail.

[0108] like Figure 6 As shown, after the industrial camera captures the original images of the conveyor belt and the nickel bar placement area, the image processing unit processes the images sequentially according to the following process:

[0109] Image scaling. The image processing unit scales the original image proportionally to reduce pixel data size and improve subsequent processing speed. The scaling factor can be flexibly configured according to actual computing resources and accuracy requirements.

[0110] Region of Interest (ROI) extraction. The image processing unit extracts a region of interest (ROI) from the scaled image, including the nickel strip and the conveyor belt boundary, based on preset coordinates. ROI extraction eliminates interference from non-target areas (such as the background outside the conveyor belt or surrounding equipment), reducing computational load. Figure 7 The image on the left shows the cropped ROI region of the nickel bar.

[0111] Bilateral filtering for noise reduction. The image processing unit performs bilateral filtering on the ROI region. Bilateral filtering is a filtering method that can reduce image noise while preserving edge features, which is crucial for the accuracy of subsequent boundary extraction.

[0112] Grayscale conversion. The filtered color image is converted into a grayscale image to prepare for subsequent binarization segmentation.

[0113] Composite binarization segmentation. This step employs a composite strategy to ensure segmentation stability. Specifically, the image processing unit first performs fixed-threshold segmentation on the grayscale image to obtain a preliminary binary image; then, it inverts this binary image so that the nickel bar target region meets the input requirement of "target is white, background is black" for the contour extraction algorithm in subsequent processing; next, it uses the inverted binary image as a mask to extract the grayscale values ​​of the corresponding nickel bar target region from the original grayscale image; finally, it performs secondary binarization on the grayscale values ​​of the target region using the Otsu automatic thresholding method to obtain the final stable binary image of the nickel bar target. Figure 7 The image on the right is the nickel bar target image after the above binarization process, where the nickel bar area is highlighted in white and the background area is suppressed in black.

[0114] Morphological processing. Morphological erosion or opening / closing operations are performed on the binary image. In this embodiment, a rectangular structuring element is used for erosion to remove small noise points and isolated interference areas, making the outer contour of the nickel bar more continuous and clear.

[0115] Contour extraction and filtering. All external contours are extracted from the processed binary image, and the area and perimeter of each extracted contour are calculated. Contours with areas smaller than preset area thresholds or perimeters smaller than preset perimeter thresholds are identified as invalid noise contours and removed; valid contours are then processed in the next step. For example... Figure 8 As shown, the effective contours retained after screening are the actual external contours of the nickel bars.

[0116] Boundary pose calculation. For the retained valid contour, the image processing unit fits it using the minimum bounding rectangle of rotation to obtain the coordinates of the four boundary vertices, the coordinates of the center point, the size information (length and width), and the rotation angle (i.e., the angle between the long side of the nickel bar and the horizontal axis of the image). Figure 8 As shown, the minimum circumscribed rectangle of revolution accurately encloses the outline of the nickel bar, with its long side pointing in the direction of the boundary line of the nickel bar. The above fitting results will be used for robot end-effector posture correction and positioning control.

[0117] When the ambient light is strong or the nickel bar surface is reflective—for example, if the nickel bar has a mirror-polished surface—the image processing unit can activate the HSV highlight area suppression module. This module converts the image to the HSV color space, identifies highlight areas based on the luminance component V, and excludes these highlight areas from the mask, thereby reducing the interference of reflections on threshold segmentation.

[0118] When further improvements in angle calculation stability are needed, the image processing unit can enable the PCA principal direction analysis module. This module calculates the principal distribution direction, i.e., the principal direction angle of the nickel bar, based on the set of contour points of the effective contour, and compares this angle with the rotation angle obtained by fitting the minimum circumscribed rotation rectangle. If the deviation between the two exceeds a preset threshold, it indicates a possible recognition anomaly, requiring manual review or re-acquiring of the image.

[0119] Through the complete software processing chain described above, visual recognition is no longer limited to simple photography or edge detection, but has formed a complete software processing flow that can directly serve robot motion control, facilitating engineering deployment and subsequent maintenance.

[0120] In this embodiment, the fusion of visual recognition results and robot control commands is implemented as follows:

[0121] After completing the above processing, the visual recognition software module not only outputs the recognition result image, but more importantly, it outputs structured data that can be used for robot closed-loop control. The specific fusion process is as follows:

[0122] The control module receives structured data output from the vision recognition software, including: the equations of the left and right boundaries of the conveyor belt (coefficients A, B, C), the equations of the left and right boundaries of the current nickel bar, the coordinates of the center point of the nickel bar, the rotation angle of the nickel bar, and the remaining distance between the left boundary of the nearest nickel bar and the left boundary of the conveyor belt. .

[0123] The control module transforms all the detection coordinates in the above image space to the robot physical coordinate system according to the pre-established hand-eye calibration transformation matrix formula (1).

[0124] Based on the aforementioned boundary parallel condition formula (2), offset distance condition formula (3), and center point collinearity calibration condition formula (4), the motion planning unit comprehensively calculates the target pose of the robot end and generates specific execution instructions, including translation (X and Y directions), rotation angle, and fine-tuning along the long side of the nickel bar.

[0125] When the visual recognition result is calculated It has entered the preset termination range, or it is estimated that continued placement will lead to... When the thickness is less than 20mm, the control module sends a stop or wait command to the robot and conveyor belt, and outputs corresponding prompts on the human-machine interface to notify the operator that the current batch of nickel bars has been arranged.

[0126] This concludes the complete description of the implementation process of the vision-guided automatic nickel bar arrangement method and system of the present invention.

[0127] Example 3

[0128] The method of the present invention will be further described in detail below using the automatic nickel bar arrangement station in a lithium battery assembly production line as an example.

[0129] In this embodiment, the processed nickel bars are 80cm long, 5cm wide, and 2cm thick. The conveyor belt is 100cm wide. The offset reference distance d0 = 5cm (50mm), the offset distance d = 2cm (20mm), and the termination distance range is 2-5cm (20-50mm). The vision system uses a 12-megapixel global shutter camera with an 8mm lens focal length, an installation height of 300mm, a field of view of 200mm × 150mm, and a pixel equivalent of 0.05mm / pixel.

[0130] Step 1: System Startup and Calibration. Perform camera hand-eye calibration to obtain pixel equivalent (0.05mm / pixel). Set the edge detection threshold and the minimum number of points for line fitting (at least 20 edge points). Set the robot's movement speed (50mm / s for low speed during placement, and 200mm / s for high speed during idle).

[0131] Step 2: Material Pickup and First Brick Placement. The robot picks up the first nickel bar from the feed tray. The vision system acquires the linear coordinates of the right boundary of the conveyor belt (represented as x=150mm, angle 0° in the robot's base coordinate system). The robot identifies the right boundary of the nickel bar; the initial gripping angle has a 1.2° deviation. The control module calculates that the nickel bar needs to be rotated -1.2° and translated to x=100mm (the right boundary of the conveyor belt x=150mm minus the offset reference distance of 50mm). The robot performs placement, and releases the magnetic attraction after confirming the position.

[0132] Step 3: Placement of the second and subsequent pieces. When placing the second piece, visually extract the left boundary coordinates of the first nickel bar (e.g., x=112mm, angle 0°). Extract the right boundary of the second nickel bar, which has a 0.8° skew. Calculate the target: rotate -0.8° and translate to x=112-20=92mm (offset 20mm to the left from the left boundary of the first piece). Simultaneously extract the center point: the Y-coordinate of the first piece's center is y=0mm (set as the baseline), and the current Y-coordinate of the second piece's center is y=+1.3mm. Therefore, an additional -1.3mm adjustment along the Y direction is needed to make the center points collinear. The final placement pose is x=92mm, y=0mm, angle 0°.

[0133] Step 4: Looping and Termination. Place the third, fourth, and so on sequentially. After each placement, measure the distance between the left edge of the current nickel bar and the left edge of the conveyor belt (x = -150mm). When the 10th bar is placed, the measured left edge x = -120mm, and the distance from the left edge of the conveyor belt is 30mm, falling within the 20-50mm range. The system checks the space required for the next nickel bar (offset 20mm). After placement, the left edge will be at -140mm, 10mm from the left edge, less than 20mm, therefore the 11th bar is not placed, and the process ends. A total of 10 nickel bars are placed, all spaced 20mm apart, with a center collinearity deviation of <0.3mm, and the end 30mm from the left edge.

[0134] Step 5: Visual Recognition Software Processing. In this embodiment, the visual recognition software runs on the control module or an industrial computer. After the industrial camera acquires the image, the software first scales the image according to a preset ratio and extracts the Region of Interest (ROI) containing the conveyor belt boundary and the nickel bar target. Then, bilateral filtering is applied to the ROI image to reduce image noise and preserve the edge features of the nickel bar. The filtered image is converted to a grayscale image. The software first performs preliminary segmentation using a fixed threshold, then inverts the binary image to ensure the nickel bar target area meets the contour extraction algorithm's input requirement of "white target, black background." Next, the software uses the binary image as a mask to extract the target grayscale region and performs secondary binarization using the Otsu automatic thresholding method to obtain a more stable binary image of the nickel bar target. For small-area noise in the binary image, the software uses rectangular structuring elements for erosion processing or opening / closing operations to reduce false contours. Finally, the software extracts the external contours and calculates the area and perimeter of each contour. When the area of ​​a contour is less than a preset area threshold or the perimeter is less than a preset perimeter threshold, it is judged as an invalid contour and discarded. For valid contours, the software uses the minimum bounding rectangle for fitting to obtain the coordinates of the four vertices, the coordinates of the center point, and the rotation angle of the nickel bar.

[0135] Step 6: Software Output and Control Linkage. The visual recognition software outputs the identified right and left boundaries, center point, rotation angle, and left and right boundary positions of the nickel bar to the motion planning unit. The motion planning unit converts these image coordinates into robot coordinates, calculates the translation, rotation angle, and fine-tuning amounts the robot needs to perform, and controls the magnetic gripper to complete the placement. If the recognition result indicates that the nickel bar is close to the left boundary of the conveyor belt, or if continuing placement will cause the remaining distance to fall below the safety threshold, the software outputs a stop flag to the control module. The control module then stops the next placement cycle and displays a termination prompt on the human-machine interface.

[0136] When the light is strong or the nickel bar surface is reflective, the software can enable the HSV high-brightness area suppression function to identify high-brightness areas through brightness components and remove them by masking. When it is necessary to further improve the stability of angle calculation, the software can enable the PCA main direction analysis function to calculate the angle of the long axis of the nickel bar according to the main distribution direction of the contour point set and compare it with the angle of the minimum circumscribed rotation rectangle for verification.

[0137] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A vision-guided automatic nickel bar arrangement method, characterized in that, Includes the following steps: S1: Using the right boundary of the conveyor belt as the positioning reference, acquire an image of the conveyor belt, extract edge points, and then combine Hough line transform to extract the right boundary line of the conveyor belt. Acquire an image of the first nickel bar, extract edge points, and then combine Hough line transform to extract the right boundary line of the first nickel bar. Adjust the pose of the first nickel bar so that the angle between its right boundary line and the right boundary line of the conveyor belt is less than or equal to a preset angle threshold, and the vertical distance between them is equal to a preset offset reference distance. Place the first nickel bar on the conveyor belt. S2: Starting from the second nickel bar, using the left boundary of the most recently placed nickel bar as the positioning reference, acquire the image of the (N-1)th nickel bar that has been placed. After extracting the edge points, combine the Hough line transform to extract the left boundary line of the (N-1)th nickel bar that has been placed. Acquire the image of the currently captured Nth nickel bar. After extracting the edge points, combine the Hough line transform to extract the right boundary line of the Nth nickel bar. Adjust the pose of the Nth nickel bar so that the angle between its right boundary line and the left boundary line of the (N-1)th nickel bar that has been placed is less than or equal to a preset angle threshold, and the vertical distance between them is equal to a preset offset distance. Place the Nth nickel bar on the conveyor belt. N is an integer greater than or equal to 2. S3: Establish a reference center line with the center point of the first nickel bar as the reference, extract the coordinates of the center point of the currently placed Nth nickel bar, calculate the axial misalignment deviation of the center point coordinates relative to the reference center line, and apply a fine adjustment amount opposite to the direction of the axial misalignment deviation along the long side of the boundary of the Nth nickel bar on the basis of satisfying the pose adjustment in S2, so that the center point of the Nth nickel bar falls on the reference center line. S4: Repeat steps S2 to S3 to place subsequent nickel bars in sequence; S5: After each placement is completed, an image of the conveyor belt is acquired, edge points are extracted, and the left boundary line of the conveyor belt is extracted using Hough transform. The vertical distance between the left boundary line of the most recently placed nickel bar and the left boundary line of the conveyor belt is calculated as the remaining distance. When the remaining distance is less than a preset termination threshold, or when the difference between the current remaining distance and the preset offset distance is used as the estimated remaining distance after the next nickel bar is placed, the placement process is stopped.

2. The visually guided automatic nickel bar arrangement method according to claim 1, characterized in that, Before executing S1, the process also includes hand-eye calibration of the industrial camera, establishing the conversion relationship between image pixel coordinates and robot physical coordinates, and converting the boundary line coordinates and center point coordinates identified in the image space to the robot physical coordinate system.

3. The visually guided automatic nickel bar arrangement method according to claim 1, characterized in that, The preset offset reference distance, the preset offset distance, and the preset termination threshold are set through a human-machine interface; the preset offset distance is less than the preset offset reference distance; and the preset angle threshold is 0.5°.

4. The visually guided automatic nickel bar arrangement method according to claim 1, characterized in that, After acquiring images in S1 and S2 but before extracting edge points, preprocessing of the acquired images is also included: Scale the original image proportionally; Extract the region of interest containing the conveyor belt boundary and the nickel bar target from the scaled image; Bilateral filtering is applied to the region of interest for noise reduction. Convert the filtered image to a grayscale image; Perform binarization segmentation on the grayscale image to obtain a binary image; Morphological processing is performed on the binary image to remove noise points.

5. The visually guided automatic nickel bar arrangement method according to claim 4, characterized in that, The step of performing binarization segmentation on the grayscale image to obtain a binary image includes: Perform fixed threshold segmentation on the grayscale image to obtain a preliminary binary image; The preliminary binary image is inverted so that the target area of ​​the nickel bar is white and the background is black; Using the inverted binary image as a mask, the grayscale value of the corresponding nickel bar target area in the grayscale image is extracted; The gray values ​​of the extracted nickel bar target area were binarized twice using the Otsu automatic thresholding method to obtain the final binary image.

6. The visually guided automatic nickel bar arrangement method according to claim 4, characterized in that, The morphological processing also includes: Extract all external contours from the binary image; Calculate the area and perimeter of each contour, and remove contours with an area smaller than a preset area threshold or a perimeter smaller than a preset perimeter threshold. The retained effective contour is fitted using the minimum bounding rectangle of rotation to obtain the boundary vertex coordinates, center point coordinates, size information, and rotation angle of the nickel bar.

7. The visually guided automatic nickel bar arrangement method according to claim 1, characterized in that, The establishment of a reference center line based on the center point of the first nickel bar, as described in S3, includes: Extract the coordinates of the center point of the first nickel bar; The reference center line is established using the coordinates of the center point as the reference point and the direction parallel to the direction of the conveyor belt movement as the reference direction.

8. The visually guided automatic nickel bar arrangement method according to claim 1, characterized in that, An interference check step is also included before each nickel bar is placed: Based on the current position and orientation of the nickel bars already placed, the boundary position of the conveyor belt, and the robot's current trajectory, determine whether the robot will collide with the already placed nickel bars or the edge of the conveyor belt when it performs the next placement action. If there is a risk of collision, adjust the robot's movement path or issue a pause command.

9. A vision-guided automatic nickel bar arrangement system, characterized in that, include: An industrial robot with an end effector equipped with grippers for grasping and releasing nickel bars; Conveyor belts are used to carry the placed nickel bars and provide an arrangement working surface; An industrial camera, mounted above the conveyor belt, is used to capture image information of the conveyor belt and the nickel bars; The control module is connected to the industrial robot, the fixture, the conveyor belt and the industrial camera respectively. The control module integrates an image processing unit and a motion planning unit, and is used to execute the vision-guided automatic nickel bar arrangement method according to any one of claims 1 to 8.

10. The vision-guided automatic nickel bar arrangement system according to claim 9, characterized in that, The clamp is a magnetic clamp, and a proximity sensor is installed on the magnetic clamp to detect whether the nickel bar is flat and adsorbed.

Images