An automated control method and system for a pick-and-place machine from calibration to placement.
By automating the acquisition and storage of calibration and placement parameters of the pick-and-place machine, and combining visual positioning and correction technology, the problems of cumbersome calibration process and human error of the pick-and-place machine are solved, and the placement accuracy and quality traceability are improved.
Patent Information
- Application Number
- CN202611558103.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-09
- Publication Date
- 2026-11-17
AI Technical Summary
The existing calibration and placement process of pick and place machines relies on manual operation, which is cumbersome and prone to human error. Incorrect parameter configuration can lead to placement position deviation or equipment malfunction.
By acquiring the calibration parameters and placement configuration parameters of the pick-and-place machine, automated control is achieved, including track calibration, top-view camera calibration, bottom-view camera calibration and flight photography calibration parameters, as well as persistent storage of axis parameters, camera parameters, input/output parameters and process timing parameters. Combined with visual positioning and correction technology, the movement of the placement head and the placement process are automatically controlled.
It has achieved automation of pick-and-place machine calibration and standardization of parameter configuration, improving placement accuracy and quality traceability, especially the pose calculation accuracy of small-sized components and high-density solder ball array components.
Smart Images

Figure CN123124328A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of pick and place machine technology, and specifically to an automated control method and system for a pick and place machine from calibration to placement. Background Technology
[0002] Currently, pick-and-place machines are the core equipment in surface mount technology (SMT) production lines. Their function is to pick up electronic components from the feeder, and after visual positioning and angle correction, accurately place them on the designated positions on the printed circuit board. The placement accuracy and efficiency of the pick-and-place machine directly affect the quality and production capacity of electronic products.
[0003] In existing technologies, the calibration and placement processes of pick-and-place machines typically rely on manual operation and experience-based parameters. During the calibration phase, operators need to manually calibrate the track width, the pixel equivalent of the upward-looking camera, the camera angle, the offset of each placement head, and the position of the flying camera. This calibration process is tedious and prone to human error. During the placement phase, operators need to manually configure axis parameters, camera parameters, input / output parameters, and process timing parameters. Incorrect parameter configuration can lead to placement position misalignment or equipment malfunction. Therefore, designing a solution to improve automated placement is a pressing technical problem that needs to be solved by those skilled in the art. Summary of the Invention
[0004] To address the aforementioned shortcomings, this invention discloses an automated control method for a pick-and-place machine from calibration to placement, which can improve the automation level of the pick-and-place machine and increase placement accuracy.
[0005] The first aspect of this invention discloses an automated control method for a pick-and-place machine from calibration to placement, comprising: Obtain the calibration parameters and mounting configuration parameters of the pick-and-place machine. The calibration parameters include track calibration parameters, top-view camera calibration parameters, bottom-view camera calibration parameters, and flight photography calibration parameters. The mounting configuration parameters include axis parameters, camera parameters, input / output parameters, and process timing parameters. After receiving the mounting start command, the control board feeding mechanism will transport the board to be mounted to the main platform and lock it. The image of the marker point on the board to be mounted will be acquired and the coordinates of the marker point will be calculated. The component data table will be read to obtain the package type, mounting coordinates and loading position of each component. Based on the coordinates of the marked points, the calibration parameters, and the mounting configuration parameters, the patch head is sequentially controlled to move to the loading position to pick up the material, and then moves to the flight photography position after picking up the material. The downward-facing camera is triggered at the flight photography position to acquire component images. The pose of the component images is calculated according to the package type to obtain angular deviation and position deviation. The rotation axis is controlled to correct the deviation according to the angular deviation. The placement head is controlled to place the component onto the board to be placed according to the corrected pose and the placement coordinates.
[0006] A second aspect of this invention discloses an automated control system for a pick-and-place machine, from calibration to placement, comprising: Acquisition module: used to acquire the calibration parameters and mounting configuration parameters of the pick and place machine. The calibration parameters include track calibration parameters, top-view camera calibration parameters, bottom-view camera calibration parameters, and flight photography calibration parameters. The mounting configuration parameters include axis parameters, camera parameters, input / output parameters, and process timing parameters. The acquisition module is used to control the board feeding mechanism to transport the board to be mounted to the main platform and lock it after receiving the mounting start command, acquire the image of the marker point on the board to be mounted and calculate the coordinates of the marker point, and read the component data table to obtain the package type, mounting coordinates and loading position of each component. Material picking module: used to control the placement head to move to the material feeding position to pick up the material in sequence according to the coordinates of the marked point, the calibration parameters and the placement configuration parameters, and then move to the flight photography position after picking up the material; Correction module: used to trigger the down-view camera to acquire component images at the flight photography position, perform pose calculation on the component images according to the package type to obtain angular deviation and position deviation, control the rotation axis to correct the deviation according to the angular deviation, and control the placement head to place the component on the board to be placed according to the corrected pose and the placement coordinates.
[0007] A third aspect of the present invention discloses an electronic device, comprising: a memory storing executable program code; a processor coupled to the memory; the processor calling the executable program code stored in the memory to execute the automated control method for calibration to placement of a pick-and-place machine disclosed in the first aspect of the present invention.
[0008] A fourth aspect of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program causes a computer to execute the automated control method for a pick-and-place machine from calibration to placement disclosed in the first aspect of the present invention.
[0009] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: The method in this embodiment of the invention solves the problem of cumbersome calibration process and easy introduction of human error in the prior art by acquiring and persistently storing calibration parameters and mounting configuration parameters, and can realize the automation of calibration and the standardization of parameter configuration. By triggering the downward-facing camera to acquire component images at the flight photography position and performing differentiated pose calculations according to package type, it solves the problem of insufficient adaptability of general image processing methods to components of different package types in the prior art, and can improve the pose calculation accuracy of small-sized components and high-density solder ball array components. By controlling the rotation axis to correct deviations according to angle deviations and recording traceability information, it improves the mounting accuracy and quality traceability capability. Attached Figure Description
[0010] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0011] Figure 1 This is a flowchart illustrating the automated control method for a pick-and-place machine from calibration to placement, as disclosed in an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of an automated control system for a pick-and-place machine from calibration to placement, provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0013] It should be noted that the terms first, second, third, fourth, etc., in the specification and claims of this invention are used to distinguish different objects, not to describe a specific order. The terms used in the embodiments of this invention include and have, and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices.
[0014] Example 1 Please see Figure 1, Figure 1 This is a flowchart illustrating the automated control method for a pick-and-place machine from calibration to placement, as disclosed in an embodiment of the present invention. The execution entity of the method described in this embodiment is an execution entity composed of software and / or hardware. This execution entity can receive relevant information via wired or / or wireless means and can send certain instructions. It may also have certain processing and storage functions. This execution entity can control multiple devices, such as remote physical servers or cloud servers and related software, or local hosts or servers and related software that perform related operations on devices located in a certain location. In some scenarios, multiple storage devices can also be controlled; these storage devices may be placed in the same location as the devices or in different locations. Figure 1 As shown, the automated control method for a pick-and-place machine from calibration to placement includes the following steps: Step S101: Obtain the calibration parameters and mounting configuration parameters of the pick-and-place machine.
[0015] The calibration parameters refer to the parameters measured and saved by the pick-and-place machine during the calibration phase, used to describe the geometric relationships and motion characteristics of the equipment. They can be understood as the reference relationship between the pick-and-place machine's vision system and motion system. The calibration parameters include track calibration parameters, top-view camera calibration parameters, bottom-view camera calibration parameters, and flight photography calibration parameters. The track calibration parameters describe the track width, moving track position, fixed track position, and baffle position. The top-view camera calibration parameters describe the pixel equivalent of the top-view camera, the camera angle, and the offset of each placement head relative to the top-view camera. The bottom-view camera calibration parameters describe the offset of each placement head of the bottom-view camera. The flight photography calibration parameters describe the Y-axis position, left X-position, right X-position, and the flight photography position of each placement head.
[0016] The placement configuration parameters refer to the various configuration parameters required for the placement machine to operate during the placement stage; they can be understood as a set of parameters for the placement machine's operation. These placement configuration parameters include axis parameters, camera parameters, input / output parameters, and process timing parameters. The axis parameters describe the pulse equivalent, stroke, starting speed, homing speed, homing offset, homing acceleration time, manual acceleration time, automatic acceleration time, stop time, and limit settings of each motion axis; the camera parameters describe the camera's serial number and pixel resolution; the input / output parameters describe input signals such as emergency stop, start / pause, homing, safety gate, track board entry, vacuum positioning detection, and feeder feeding completion, as well as output signals such as indicator lights, buzzers, board present, board requested, air blowing, air suction, and feeder feeding; the process timing parameters describe the Mark scan positioning time, track direction, track descent time, track positioning time, track side push time, track lifting time, track holding time, board entry speed, board exit speed, number of pick-up retries, pre-pick-up delay, pre-discard delay, vacuum opening delay, pick-up visual time, Z-axis pick-up speed, feeder time, pre-placement delay, Z-axis placement delay, placement vacuum closing delay, placement vacuum breaking delay, and Z-axis placement speed.
[0017] In one embodiment, the calibration parameters and the placement configuration parameters are persistently stored in a configuration file. Specifically, after the pick-and-place machine is powered on, the host computer reads the configuration file and loads the calibration parameters and placement configuration parameters into memory for use in subsequent placement processes. When a new calibration result is obtained during the calibration phase, the host computer writes the new calibration result into the configuration file, thus achieving the persistence of the calibration result.
[0018] Specifically, track calibration parameters can include the coordinates of the active track position (e.g., X=150.00mm, Y=120.00mm), the coordinates of the fixed track position (e.g., X=150.00mm, Y=80.00mm), the track width (e.g., 40.00mm), and the baffle position (e.g., X=200.00mm). The upward-looking camera calibration parameters can include pixel equivalent (e.g., 0.01mm / pixel), camera angle (e.g., 0.5°), and offset of each patch head (e.g., patch head 1 offset X=0.05mm, Y=-0.03mm). Flight photography calibration parameters can include the flight photography Y-axis position (e.g., Y=200.00mm), left X position (e.g., X=100.00mm), and right X position (e.g., X=300.00mm). By acquiring and persistently storing the calibration parameters and mounting configuration parameters, a unified parameter benchmark is provided for subsequent mounting processes, avoiding the tedious operation of recalibrating and configuring before each mounting.
[0019] Step S102: After receiving the mounting start command, control the board feeding mechanism to transfer the board to be mounted to the main platform and lock it, collect the image of the marker points on the board to be mounted and calculate the coordinates of the marker points, and read the component data table to obtain the package type, mounting coordinates and loading position of each component.
[0020] The placement start command refers to the control command triggered by the operator through the host computer interface or external signals to start the placement process. The board feeding mechanism refers to the mechanical mechanism in the pick-and-place machine used to transport the board to be placed from the board feeding end to the main platform, and may include a belt conveyor mechanism, a baffle positioning mechanism, a lifting mechanism, and a clamping mechanism. The main platform refers to the work platform in the pick-and-place machine used to carry the board to be placed and perform the placement operation. The marker point refers to the pre-set reference mark on the board to be placed for visual positioning, usually circular or cross-shaped. The component data table refers to the table storing the placement information of each component, and may include fields such as component identifier, package type, placement coordinates, loading position, and placement head identifier.
[0021] In one embodiment, the process of controlling the board feeding mechanism to transport and lock the board to be mounted to the main platform includes: the host computer controlling the belt conveyor mechanism of the board feeding mechanism to start, transporting the board to be mounted from the board feeding end to the main platform; after the board to be mounted arrives at the main platform, the baffle positioning mechanism positions the board to be mounted.
[0022] In one embodiment, the process of acquiring images of marker points on the board to be mounted and calculating their coordinates includes: the host computer controlling a top-view camera to move above the marker point; triggering the top-view camera to take a picture and waiting for the Mark scanning positioning time; calling a vision algorithm to process the marker point image, locate the marker point, and calculate its coordinates. The marker point coordinates are used for subsequent mounting position compensation, i.e., calculating the actual movement position of the mounting head based on the marker point coordinates and the mounting coordinates in the component data sheet. Specifically, if there are two marker points on the board to be mounted, with the first marker point having coordinates of (X=10.00mm, Y=10.00mm) and the second marker point having coordinates of (X=200.00mm, Y=150.00mm), and the mounting coordinates of a component in the component data sheet being (X=100.00mm, Y=80.00mm), then the actual mounting position of the component can be calculated based on the relative relationship between the marker point coordinates and the mounting coordinates. Through board entry locking, marker point recognition, and component data sheet reading, accurate positioning references and component information are provided for subsequent material handling and mounting.
[0023] Step S103: Based on the coordinates of the marked points, the calibration parameters, and the mounting configuration parameters, control the patch head to move to the feeding position to pick up the material, and then move to the flight photography position after picking up the material.
[0024] The loading position refers to the location of the component in the feeder, sleeve, or tray, which can be understood as the position where the placement head descends along the Z-axis when picking up the component. The flying photography position refers to the position where the placement head moves after picking up the component for the downward-looking camera to take pictures, which can be understood as the position where the placement head delivers the component into the downward-looking camera's field of view.
[0025] In one embodiment, the process of controlling the placement head to move to the loading position for material picking includes: the host computer controlling the placement head to move to the loading position in the XY plane according to the loading position in the component data sheet; controlling the Z-axis to descend to the picking height; activating the vacuum and delaying the vacuum pick-up process; controlling the Z-axis to rise to a safe height; and detecting the vacuum arrival signal. When the vacuum arrival signal is valid, material picking is considered successful; when the vacuum arrival signal is invalid, material picking is considered failed, and the number of retries is accumulated. If the number of retries is less than the preset number of retries, material picking is attempted again; otherwise, material is discarded and the result is recorded.
[0026] In one embodiment, the process of controlling the placement head to move to the flight photography position after material handling includes: the host computer controlling the placement head to move to the flight photography position according to the flight photography calibration parameters of each placement head. Specifically, the placement head is controlled to move to the flight photography position in the XY plane, and the Z-axis and R-axis are controlled to move to the Z-axis and R-axis positions corresponding to the flight photography position.
[0027] For example, if the flight photography calibration parameters for the placement head 1 are X=100.00mm, Y=200.00mm, Z=5.00mm, and R=0°, then the placement head 1 is moved to that position to wait for the downward-looking camera to take a picture. By moving the pick-up and flight photography positions, the component is taken out from the feeder and sent into the downward-looking camera's field of view, providing an image basis for subsequent visual correction.
[0028] Step S104: Trigger the downward-facing camera to acquire component images at the flight photography position, perform pose calculation on the component images according to the package type to obtain angle deviation and position deviation, control the rotation axis to correct the deviation according to the angle deviation, and control the placement head to place the component on the board to be placed according to the corrected pose and the placement coordinates.
[0029] The downward-facing camera refers to a camera mounted on the pick-and-place machine platform with its lens facing upwards, used to photograph the components held by the pick-and-place head from below. The package type refers to the component's package form, including first-class packages (such as CHIP, SOP, etc.) and second-class packages (such as BGA, etc.). The pose calculation refers to calculating the component's angle and center position in the image using a visual algorithm. The angle deviation refers to the difference between the component's actual angle and the target angle. The position deviation refers to the difference between the component's actual center position and the target center position. The rotation axis refers to the R-axis used to drive the pick-and-place head to rotate. Correction refers to controlling the rotation axis to rotate according to the angle deviation, ensuring the component angle matches the target angle.
[0030] In one embodiment, the process of triggering the downward-looking camera to acquire component images includes: controlling the placement head to move to the flight photography position according to the flight photography calibration parameters; triggering the downward-looking camera to take a picture and waiting for the material picking visual time; calling the visual pose calculation and obtaining the angle deviation; controlling the R-axis rotation to correct the deviation according to the angle deviation; and writing the angle deviation and correction result into the operation log. The material picking visual time is used to wait for the camera exposure, image transmission, illumination stabilization, and mechanical vibration attenuation to complete, ensuring that subsequent visual calculations use a clear image of the current component.
[0031] In one embodiment, the process of calculating the pose of a component image based on the package type includes: when the package type is a first type of package, the component image is sequentially smoothed, corner detection is performed, and interference points are filtered out; sub-pixel coordinates are calculated based on the filtered corner points; and the component angle and center offset are calculated based on the sub-pixel coordinates. When the package type is a second type of package, the component image is sequentially smoothed and thresholded; the outline of each solder ball is extracted and the sub-pixel coordinates are calculated; after circle fitting of each solder ball, key solder balls are selected; and the component angle and center offset are calculated based on the center of the key solder balls.
[0032] In one embodiment, the process of controlling the rotating axis for correction based on angular deviation includes: when the package type is a first type, calculating the number of pulses based on the angular deviation and driving the stepper motor to rotate the corresponding number of steps to complete the correction; when the package type is a second type, inputting the angular deviation as a position setpoint into the position adjustment loop, which then generates a switch control signal via the speed adjustment loop and the current adjustment loop to drive the brushless motor to rotate and complete the correction; recording the actual angle after correction and comparing it with the angular deviation; if the comparison result exceeds a preset range, it is determined that a second correction is needed. By using visual pose calculation and rotating axis correction, the component angular deviation and position deviation are compensated to the mounting position, improving mounting accuracy.
[0033] In this embodiment of the invention, calibration automation and parameter configuration standardization are achieved by acquiring and persistently storing calibration parameters and placement configuration parameters; placement automation is achieved through complete process control including board entry locking, marker point recognition, material picking, flying photography, visual correction, and placement; and placement accuracy of components with different package types is improved by performing differentiated pose calculation and correction according to package type.
[0034] Specifically, the step of sequentially controlling the patch head to move to the loading position for material handling includes: Control the patch head to move in the XY plane to the loading position, control the Z axis to descend to the picking height, turn on the vacuum and press the pick-up vacuum start delay to wait; Control the Z-axis to rise to a safe height and detect the vacuum positioning signal; When the vacuum signal is valid, the material is determined to be successfully retrieved. When the vacuum signal is invalid, the material is determined to be retrieved to fail and the number of retries is accumulated. When the number of retries is less than the preset number of retries, the material is retrieved again. Otherwise, the material is discarded and recorded.
[0035] More preferably, obtaining the calibration parameters of the pick-and-place machine includes: Step S1011: Control the width adjustment axis to return to zero and move it to the position of the active track. Obtain the position coordinates of the active track and the position coordinates of the fixed track respectively. Calculate the track width based on the position coordinates of the active track and the position coordinates of the fixed track, and calibrate the position of the baffle.
[0036] The width-adjusting axis refers to the motion axis used to adjust the track width, which can be understood as the axis that drives the movable track to move. The movable track position coordinates refer to the position coordinates of the movable track in the Y direction. The fixed track position coordinates refer to the position coordinates of the fixed track in the Y direction. The track width refers to the distance between the movable track and the fixed track.
[0037] In one embodiment, the process of controlling the width adjustment axis to return to zero and move to the active track position includes: the host computer controlling the width adjustment axis to perform a zero-return action; after the zero-return is completed, controlling the width adjustment axis to move to the active track position; reading the active track position coordinates and the fixed track position coordinates from the motion control card; and calculating the track width based on the active track position coordinates and the fixed track position coordinates. The track width = active track position coordinates - fixed track position coordinates.
[0038] Specifically, if the position coordinates of the movable track are Y=120.00mm and the position coordinates of the fixed track are Y=80.00mm, then the track width = 120.00 - 80.00 = 40.00mm. The baffle position calibration can be completed by controlling the baffle to move to a preset position and then reading its position coordinates. Through track calibration, the positional relationship between the movable track, fixed track, and baffle is determined, providing a positional reference for the entry and exit of the baffle.
[0039] Step S1012: Control the upward-looking camera to acquire images at the calibrated position, calculate the pixel equivalent and camera angle based on the positioning coordinates of the two photos taken before and after moving a preset distance, and calibrate the offset of each patch head relative to the upward-looking camera.
[0040] The upward-viewing camera refers to a camera mounted above the pick-and-place machine with its lens facing downwards, used to capture images of the marking points on the board to be placed from above. The pixel equivalent refers to the actual distance corresponding to one pixel in the physical world. The camera angle refers to the rotation angle of the camera coordinate system relative to the motion mechanism coordinate system. The offset of each placement head relative to the upward-viewing camera refers to the positional deviation between the center of each placement head and the center of the upward-viewing camera.
[0041] In one embodiment, the process of calculating pixel equivalent and camera angle includes: acquiring an image at a calibrated position and locating a marker point to obtain first positioning coordinates; controlling a motion mechanism to move a preset distance along a set direction and then acquiring an image again and locating the same marker point to obtain second positioning coordinates; calculating pixel equivalent based on the ratio of the pixel distance between the first positioning coordinates and the second positioning coordinates to a preset distance; and calculating the camera angle based on the coordinate difference between the first positioning coordinates and the second positioning coordinates.
[0042] In one embodiment, the process of calibrating the offset of each patch head relative to the upward-looking camera includes: moving the patch head to the calibration position and recording the patch head position; moving the upward-looking camera to the calibration position and recording the camera position; aligning the camera center with a marker point and recording the alignment position; and calculating the offset based on the patch head position and the alignment position. Specifically, if the preset movement distance is 10.00 mm, the first positioning coordinate is (500, 500) pixels, and the second positioning coordinate is (1500, 500) pixels, then the pixel distance = 1000 pixels, and the pixel equivalent is 0.01 mm / pixel. If the coordinate difference is (1000, 20), then the camera angle ≈ 1.15°. Through the upward-looking camera calibration, the pixel equivalent, camera angle, and offset of each patch head are determined, providing a reference for subsequent marker point positioning and patch position compensation.
[0043] Step S1013: Control the downward-facing camera to trigger the taking of pictures and locate the center of the component. After aligning the center of the component with the center of the downward-facing camera, record the first alignment position. Control the patch head to rotate by a preset angle and align it again and record the second alignment position. Calculate the offset of each patch head relative to the downward-facing camera based on the first alignment position and the second alignment position.
[0044] The downward-facing camera refers to a camera mounted on the placement machine platform with its lens facing upwards, used to photograph the component being held by the suction head from below. The first alignment position refers to the position of the placement head when the center of the component is aligned with the center of the downward-facing camera. The second alignment position refers to the position of the placement head when, after rotating the placement head by a preset angle, the center of the component is aligned with the center of the downward-facing camera again. The preset angle can be 180°.
[0045] In one embodiment, the process of calculating the offset of each patch head relative to the downward-looking camera includes: triggering the downward-looking camera to take a picture and locate the center of the component; aligning the center of the component with the center of the downward-looking camera and recording a first alignment position; controlling the patch head to rotate 180° and then aligning again and recording a second alignment position; calculating the offset based on the first alignment position and the second alignment position. The offset = (first alignment position + second alignment position) / 2.
[0046] Specifically, if the first alignment position is X=100.00mm and Y=200.00mm, and the second alignment position is X=100.10mm and Y=200.10mm, then the offset X=(100.00+100.10) / 2=100.05mm, and the offset Y=(200.00+200.10) / 2=200.05mm. Through downward camera calibration, the offset of each patch head relative to the downward camera was determined, providing a coordinate transformation reference for subsequent flight photography correction.
[0047] Step S1014: Calibrate the Y-axis position, left X position, right X position, and the flight photography position of each patch head, and write the calibration results into the configuration file.
[0048] The Y-axis position for flight photography refers to the Y-axis position when the patch head moves to the flight photography position. The left X-position and right X-position refer to the X-axis positions of the left and right suction heads of the patch head during flight photography. The flight photography position of each patch head refers to the X, Y, Z, and R-axis positions of each patch head during flight photography.
[0049] In one embodiment, the process of calibrating the flight image position includes: setting flight image parameters; calibrating the Y-axis position of the flight image; calibrating the left X-position of the flight image; calibrating the right X-position of the flight image; calibrating the flight image position of each patch head; and writing the calibration results into a configuration file. Through flight image calibration, the flight image position of each patch head is determined, providing a positional reference for subsequent flight image correction.
[0050] More preferably, the step of calculating the pixel equivalent and camera angle based on the positioning coordinates of two photos taken before and after moving a preset distance includes: Step S10121: Acquire an image at the calibration location and locate the marker point to obtain the first positioning coordinates.
[0051] The calibration position refers to a pre-defined location used for acquiring calibration images. The marker point is a reference mark on the calibration plate used for visual positioning. The first positioning coordinates refer to the pixel coordinates of the marker point in the image.
[0052] In one embodiment, the process of acquiring an image and locating a marker point at a calibrated location includes: controlling a motion mechanism to move to the calibrated location; triggering the upward-looking camera to take a picture; calling a visual algorithm to process the image, locate the marker point, and calculate the first positioning coordinates. If the calibrated location is X=100.00mm and Y=100.00mm, the first positioning coordinates of the marker point obtained after the upward-looking camera takes a picture are (500, 500) pixels.
[0053] Step S10122: After controlling the motion mechanism to move a preset distance along the set direction, the image is acquired again and the same marker point is located to obtain the second positioning coordinates.
[0054] The preset distance refers to the physical distance the motion mechanism moves. The set direction refers to the direction in which the motion mechanism moves, which can be the X or Y direction. The second positioning coordinates refer to the pixel coordinates of the marker point in the image after the point has moved.
[0055] In one embodiment, the process of controlling the motion mechanism to move a preset distance along a set direction and then acquiring an image and locating the same marker point again includes: controlling the motion mechanism to move a preset distance along the X direction; triggering the upward-looking camera to take a picture; calling a visual algorithm to process the image, locating the same marker point, and calculating the second positioning coordinates. If the preset distance is 10.00mm, after the motion mechanism moves 10.00mm along the X direction, the upward-looking camera takes a picture and locates the same marker point, obtaining the second positioning coordinates as (1500, 520) pixels.
[0056] Step S10123: Calculate the pixel equivalent based on the ratio of the pixel distance between the first positioning coordinate and the second positioning coordinate to the preset distance.
[0057] Wherein, the pixel distance refers to the number of pixels the marker point moves in the image. The pixel equivalent = preset distance / pixel distance. In one embodiment, the process of calculating the pixel equivalent includes: calculating the pixel distance between the first positioning coordinates and the second positioning coordinates; dividing the preset distance by the pixel distance to obtain the pixel equivalent.
[0058] Step S10124: Calculate the camera angle based on the coordinate difference between the first positioning coordinate and the second positioning coordinate, and save the pixel equivalent and the camera angle to the upward camera calibration parameters.
[0059] The coordinate difference refers to the difference between the first positioning coordinate and the second positioning coordinate. The camera angle is calculated as arctan(Δv / Δu) × 180 / π.
[0060] In one embodiment, the process of calculating the camera angle includes: calculating the coordinate difference between the first positioning coordinate and the second positioning coordinate; calculating the camera angle based on the coordinate difference; and saving the pixel equivalent and the camera angle to the upward-looking camera calibration parameters. By calculating the pixel equivalent and the camera angle, a transformation relationship between the image coordinate system and the motion mechanism coordinate system is established, providing a benchmark for subsequent mounting position compensation.
[0061] More preferably, the step of performing pose calculation on the component image according to the package type includes: Step S1041: When the package type is the first type of package, the component image is smoothed, corner detection and interference point filtering are performed in sequence. Sub-pixel coordinates are calculated based on the filtered corners, and the component angle and center offset are calculated based on the sub-pixel coordinates.
[0062] The first type of package refers to packages with obvious corner features, such as CHIP and SOP. The smoothing process refers to denoising and edge enhancement of the component image. The corner detection refers to detecting the corner positions in the component image. The interference point filtering refers to filtering out corners that do not meet the requirements according to the four-corner criterion. The sub-pixel coordinates refer to the non-integer coordinates of the corners. The component angle refers to the deviation of the component from the target angle. The center offset refers to the deviation of the component center from the target center.
[0063] In one embodiment, the process of sequentially smoothing, detecting corners, and filtering out interference points in a component image includes: constructing an optimization framework constrained by gradient sparsity, smoothing the component image using gradient minimization within the optimization framework; detecting corners in the smoothed component image to obtain candidate corners; and filtering out interference points according to the four-corner criterion to obtain valid corners.
[0064] Specifically, if the component image is an SOP package, after smoothing, candidate corner points are detected. Four valid corner points are selected according to the four-corner criteria (minimum Y, minimum X, maximum Y, maximum X). Sub-pixel coordinates are calculated based on the valid corner points, and the component angle and center offset are calculated based on the sub-pixel coordinates.
[0065] Step S1042: When the package type is the second type of package, the component image is smoothed and thresholded sequentially, the outline of each solder ball is extracted and the sub-pixel coordinates are calculated, the key solder balls are selected after circle fitting of each solder ball, and the component angle and center offset are calculated based on the center of the key solder balls.
[0066] The second type of package refers to packages with solder ball array features, such as BGA. Threshold segmentation refers to the binarization of the component image. The solder ball contour refers to the boundary of the solder ball in the binarized image. Circle fitting refers to the process of fitting a circle based on the solder ball contour point set. The key solder balls refer to the four corner solder balls used to calculate the component's angle and center offset.
[0067] Specifically, if the component image is a BGA package, after smoothing, threshold segmentation is used to obtain a binarized image. The contours of each solder ball are extracted and the sub-pixel coordinates are calculated. Circle fitting is performed on each solder ball to obtain the center and radius. Four corner solder balls are selected as key solder balls. The component angle and center offset are calculated based on the center of the key solder balls.
[0068] Step S1043: Output the calculated element angle as the angle deviation.
[0069] The angular deviation refers to the difference between the actual angle of the component and the target angle. This angular deviation is used for subsequent rotational axis correction.
[0070] Step S1044, the step of controlling the rotating shaft to correct the deviation according to the angle deviation, includes: when the package type is the first type of package, calculating the number of pulses according to the angle deviation and driving the stepper motor to rotate the corresponding number of steps to complete the correction; when the package type is the second type of package, inputting the angle deviation as a position setpoint into the position adjustment ring, and generating a switch control signal through the speed adjustment ring and the current adjustment ring in sequence to drive the brushless motor to rotate to complete the correction; recording the actual angle after correction and comparing it with the angle deviation, and determining that a second correction is needed when the comparison result exceeds the preset range.
[0071] The pulse count refers to the number of pulses required for the stepper motor to rotate. The position adjustment loop, speed adjustment loop, and current adjustment loop constitute a three-closed-loop control structure. The secondary correction refers to the need for a second correction when the difference between the actual angle after correction and the angle deviation exceeds a preset range.
[0072] In one embodiment, the process of controlling the rotating shaft to correct the deviation based on the angle deviation includes: when the package type is a first type of package, calculating the number of pulses based on the angle deviation and driving the stepper motor to rotate the corresponding number of steps; when the package type is a second type of package, inputting the angle deviation as a position setpoint into the position adjustment ring, and generating a switch control signal through the speed adjustment ring and the current adjustment ring in sequence to drive the brushless motor to rotate; recording the actual angle after correction and comparing it with the angle deviation.
[0073] More preferably, triggering the downward-facing camera to acquire images at the flight photography location includes: Step S104a1: Control the patch head to move to the flight photography position according to the flight photography calibration parameters.
[0074] The flight photography calibration parameters are parameters determined during the calibration phase to describe the flight photography position. The flight photography position refers to the position the patch head moves to after picking up the material, used for taking pictures with the downward-looking camera. In one embodiment, the process of controlling the patch head to move to the flight photography position according to the flight photography calibration parameters includes: controlling the patch head to move to the flight photography position in the XY plane according to the flight photography positions of each patch head in the flight photography calibration parameters, and controlling the Z-axis and R-axis to move to the Z-axis and R-axis positions corresponding to the flight photography position.
[0075] Step S104a2: Trigger the downward-facing camera to take a picture and wait for the material picking visual time.
[0076] The material handling vision time refers to the waiting time after the downward-facing camera is triggered to take a picture, which is used to wait for the camera to expose, transmit the image, stabilize the light, and allow mechanical vibration to decay. The typical value of the material handling vision time is from a few milliseconds to tens of milliseconds.
[0077] In one embodiment, the process of triggering the downward-facing camera to take a picture and waiting according to the material picking visual time includes: sending a trigger signal to the camera to initiate an exposure and image acquisition; waiting according to the material picking visual time; and reading the image after the waiting period is completed. Specifically, if the material picking visual time is 10ms, then after triggering the downward-facing camera to take a picture, wait for 10ms to complete the camera exposure and image transmission before reading the image.
[0078] Step S104a3: Call the visual pose calculation and obtain the angle deviation, and control the R-axis rotation to correct the deviation according to the angle deviation.
[0079] The visual pose calculation refers to the process of calculating the component angle and center offset using a visual algorithm. The R-axis rotation correction refers to controlling the R-axis rotation based on the angle deviation to make the component angle consistent with the target angle. In one embodiment, the process of calling the visual pose calculation and obtaining the angle deviation includes: calling the visual algorithm to process the component image to obtain the component angle and center offset; outputting the component angle as the angle deviation; and controlling the R-axis rotation correction based on the angle deviation.
[0080] Step S104a4: Write the angle deviation and correction result into the operation log, and record the log and skip the current element or take a new picture when the visual calculation fails.
[0081] The operation log refers to a log file used to record various events and data during the mounting process. The visual calculation failure refers to the visual algorithm failing to calculate valid angular and positional deviations. Re-taking the photo refers to re-triggering the downward-facing camera to take another picture and re-calculating the visual pose.
[0082] In one embodiment, the process of writing the angle deviation and correction result into the operation log includes: recording the component identifier, package type, chip header identifier, angle deviation, actual angle after correction, and whether the correction was successful; when the difference between the actual angle after correction and the angle deviation is greater than a preset deviation threshold, it is determined that a second correction is required and recorded.
[0083] In one embodiment, the process of logging and skipping the current component or re-taking the image when visual computation fails includes: accumulating the number of failures when visual computation fails; when the number of failures is less than a preset number of retakes, re-triggering the downward-looking camera to take another picture and recalculating the visual pose; when the number of failures reaches the preset number of retakes, logging is recorded and the current component is skipped, and the placement head is controlled to discard the component. By triggering the downward-looking camera to acquire component images through the flight photography position, visual correction after component removal is achieved, and the correction process and results are recorded through the operation log.
[0084] More preferably, triggering the downward-facing camera to acquire images at the flight photography location includes: Step S104b1: During the movement of the suction head, the motion control card reads the position of the suction head encoder in real time.
[0085] The motion control card refers to the control card used in the pick-and-place machine to control the movement of each motion axis. The pick-and-place head encoder refers to the encoder installed on the motion axis of the pick-and-place head, used to provide real-time feedback on the position of the pick-and-place head.
[0086] Step S104b2: Compare the encoder position with the flight photography position.
[0087] The flight photography position is a position determined in a specific phase to trigger the flight camera to take a picture. In one embodiment, the process of comparing the encoder position with the flight photography position includes: comparing the current suction head position with the flight photography position; when the suction head position enters the flight photography position, it is determined that the triggering condition is met.
[0088] Step S104b3: When the encoder position enters the flight photography position, the motion control card generates a trigger signal and sends it to the flight camera and the strobe light source; after receiving the trigger signal, the flight camera exposes in a global shutter mode, and the strobe light source illuminates for a duration of microseconds during the exposure to obtain an image of the flying material.
[0089] The trigger signal refers to the electrical signal generated by the motion control card used to trigger the camera to take a picture and the light source to light up. The global shutter mode refers to a method where the camera sensor simultaneously exposes all pixels, avoiding motion distortion caused by rolling shutters. The strobe source refers to a light source that illuminates for a duration on the order of microseconds during exposure. The flying material image refers to an image of the material captured during the movement of the suction head.
[0090] In one embodiment, when the encoder position enters the flight photography position, the process of the motion control card generating a trigger signal and sending it to the flight camera and stroboscopic light source includes: the motion control card generating the trigger signal; sending the trigger signal to the flight camera and stroboscopic light source; the flight camera exposing in a global shutter mode after receiving the trigger signal; the stroboscopic light source illuminating for a duration in microseconds during the exposure; and obtaining an image of the flying material. By directly triggering the flight camera to take pictures based on the encoder position using the motion control card, photography during the suction head movement is achieved, improving the placement cycle time.
[0091] More preferably, after obtaining the image of the flying material, the method further includes: Step S104c1: During the movement of the suction head, the encoder collects the suction head position data in real time, and the inertial measurement unit collects the camera vibration data in real time.
[0092] The inertial measurement unit (IMU) refers to a sensor mounted on the camera for measuring camera vibration data. The camera vibration data refers to the camera's vibration displacement and velocity during exposure.
[0093] In one embodiment, the process of real-time acquisition of suction head position data by an encoder and real-time acquisition of camera vibration data by an inertial measurement unit includes: the encoder acquiring suction head position data at a preset sampling period; the inertial measurement unit acquiring camera vibration data at a preset sampling period; and storing the suction head position data and camera vibration data in a cache.
[0094] Step S104c2: Construct a point spread function based on the suction head position data and the camera vibration data. The point spread function characterizes the motion trajectory of the material on the image sensor during exposure.
[0095] The point spread function is a mathematical function describing the response of an imaging system to a point light source; it can be understood as the trajectory of the material on the image sensor during exposure. The point spread function is constructed based on suction head position data and camera vibration data.
[0096] In one embodiment, the process of constructing a point spread function based on suction head position data and camera vibration data includes: calculating the material movement velocity during exposure based on suction head position data; calculating the camera vibration displacement during exposure based on camera vibration data; and constructing a point spread function based on the movement velocity and vibration displacement.
[0097] Step S104c3: Perform deconvolution operation on the flying material image according to the point spread function to restore the material outline and obtain a clear material image.
[0098] Specifically, the point spread function is: ,in, Let represent the intensity distribution of the point spread function at coordinates (x, y) on the image sensor. It is the change of the relative displacement of the material with respect to the image sensor in the x and y directions over time during the exposure period; Let T be the impulse function and T be the exposure time. This is the normalization coefficient.
[0099] The deconvolution operation refers to the inverse operation performed on a blurred image based on the point spread function. The clear material image refers to the material image with clear outlines recovered after the deconvolution operation.
[0100] In one embodiment, the process of performing deconvolution operation on the image of flying material based on the point spread function includes: calculating a blur level index based on the point spread function; comparing the blur level index with a preset blur threshold; performing deconvolution operation on the image of flying material when the blur level index is greater than the preset blur threshold; and skipping the deconvolution operation and directly using the image of flying material as a clear image when the blur level index is less than or equal to the preset blur threshold.
[0101] Specifically, if the blur level index is 0.8 and the preset blur threshold is 0.5, then the blur level index is greater than the preset blur threshold, and deconvolution operation is performed on the image of the flying material. If the blur level index is 0.3 and the preset blur threshold is 0.5, then the blur level index is less than the preset blur threshold, and the deconvolution operation is skipped. By performing deconvolution operation on the image of the flying material based on the point spread function, motion blur is eliminated, and a clear image of the material is obtained.
[0102] More preferably, after the downward-facing camera is triggered to acquire the element image at the flight photography position, the image is further smoothed, specifically including: An optimization framework constrained by gradient sparsity is constructed, in which gradient minimization is used to smooth the component image; the smoothing process is controlled by a smoothing parameter and a scaling parameter, wherein the smoothing parameter controls the degree of smoothing and the scaling parameter controls the edge preservation ratio.
[0103] The gradient sparsity refers to the small number of pixels with non-zero gradients in the image. The optimization framework transforms the image smoothing problem into a mathematical problem of finding the optimal solution. Gradient minimization refers to iteratively minimizing the number of non-zero gradients. The smoothing parameter controls the smoothing degree and ranges from 0.001 to 0.1. The scaling parameter controls the edge preservation ratio and ranges from 1 to 2.
[0104] In one embodiment, an optimization framework constrained by gradient sparsity is constructed. The process of smoothing the component image using gradient minimization within this framework includes: constructing an optimization framework containing an objective term and constraint terms. The objective term constrains the difference between the smoothed component image and the original component image, while the constraint terms constrain the number of pixels with non-zero gradients in the smoothed component image; iteratively calculating the gradient of each pixel in the component image; in each iteration, setting the gradient of pixels with gradient magnitudes less than a preset gradient threshold to zero and updating the corresponding pixel's grayscale value; repeating the iteration until a preset convergence condition is met. If the smoothing parameter is 0.01 and the scaling parameter is 1.5, the component image is smoothed to eliminate background texture and noise from the suction head, while preserving and enhancing the component edges.
[0105] The smoothing and scaling parameters are adjusted based on the nozzle background grayscale, component size, and lighting conditions in a real production environment. Here, the nozzle background grayscale refers to the grayscale value of the nozzle in the image. The component size refers to the pixel size of the component in the image. The lighting conditions refer to the light intensity and uniformity when the component image was captured.
[0106] In one embodiment, the process of adjusting the smoothing parameter and scaling parameter based on the nozzle background grayscale, component size, and lighting conditions in a real production environment includes: increasing the smoothing parameter to enhance background removal when the difference between the nozzle background grayscale and the component edge grayscale is less than a preset grayscale threshold; decreasing the smoothing parameter to avoid over-smoothing of the component edges when the component size is less than a preset size threshold; and increasing the scaling parameter to enhance the preservation of the main edges of the component when the lighting conditions are highly reflective. Specifically, if the nozzle background grayscale is 100, the component edge grayscale is 120, and the preset grayscale threshold is 30, then the grayscale difference of 20 is less than the preset grayscale threshold of 30, and the smoothing parameter is increased to 0.05 to enhance the background removal.
[0107] The smoothed component image is used as input for subsequent pose calculation. The pose calculation refers to the process of calculating the component angle and center offset using a visual algorithm.
[0108] In one embodiment, the process of using the smoothed component image as input for subsequent pose calculation includes: performing binarization segmentation on the smoothed component image; performing edge detection based on the binarization segmentation results to obtain edge pixel candidate points; and performing sub-pixel coordinate estimation and pose calculation based on the edge pixel candidate points. Smoothing with gradient sparsity constraints eliminates the suction head background and noise, preserves and enhances component edges, and provides a clear image foundation for subsequent pose calculation.
[0109] In this embodiment of the invention, mean filtering and Gaussian filtering are not used to denoise the input image. Mean filtering and Gaussian filtering have limited denoising capabilities and cannot effectively eliminate the interference of the patch header background on component imaging. In particular, Gaussian filtering can cause image blurring, increasing the difficulty of subsequent feature extraction and calculation. In contrast, the smoothing processing method described in this embodiment of the invention can suppress the interference of the patch header background to the greatest extent. While preserving the main feature information of the image, it also enhances the feature edges to a certain extent, which is more conducive to subsequent feature extraction and pose calculation.
[0110] More specifically, the adjustment of the smoothing parameter, or proportional parameter, includes: When the difference between the background grayscale of the nozzle and the grayscale of the component edge is less than a preset grayscale threshold, the smoothing parameter is increased to enhance the degree of background elimination; When the component size is smaller than a preset size threshold, the smoothing parameter is reduced to avoid excessive smoothing of the component edges; When the lighting conditions are strong reflective, the scaling parameter is increased to enhance the preservation of the main edges of the component; the adjusted smoothing parameter and the scaling parameter are recorded in the mounting configuration parameters for subsequent smoothing processing of the component image.
[0111] In this embodiment of the invention, the smoothing parameters and scaling parameters are adaptively adjusted based on real production environment factors such as the difference between the background grayscale of the nozzle and the grayscale of the component edge, the component size, and the lighting conditions. This solves the problem of inconsistent smoothing effects under different production environments caused by fixed smoothing parameters in the prior art. It can enhance background elimination when there is strong background interference, avoid excessive edge smoothing when there are small-sized components, and strengthen edge preservation when there is strong reflection. The adjusted parameters are recorded in the placement configuration parameters for subsequent use, which improves the adaptability of the pick-and-place machine's vision correction system to different production environments and the consistency of placement accuracy.
[0112] More preferably, the step of performing pose calculation on the component image according to the package type includes: Step S104d1: Statistically analyze the grayscale distribution of the component image, set a segmentation threshold to divide the pixels of the component image into a foreground region and a background region, and calculate the average grayscale of the foreground region, the average grayscale of the background region, the pixel ratio of the foreground region, the pixel ratio of the background region, and the overall average grayscale of the component image.
[0113] The grayscale distribution refers to the number of pixels with each grayscale value in the component image. The segmentation threshold refers to the grayscale value used to divide pixels into foreground and background. The foreground region refers to the region where the component is located. The background region refers to the region where the suction head background is located. The pixel ratio of the foreground region refers to the proportion of pixels in the foreground region to the total number of pixels. The overall average grayscale refers to the average grayscale of all pixels in the component image.
[0114] In one embodiment, the process of statistically analyzing the grayscale distribution of a component image and setting a segmentation threshold to divide the pixels of the component image into foreground and background regions includes: counting the number of pixels with each grayscale value in the component image; setting a segmentation threshold; classifying pixels with grayscale values greater than the segmentation threshold as foreground regions and pixels with grayscale values less than or equal to the segmentation threshold as background regions; and calculating the average grayscale of the foreground region, the average grayscale of the background region, the pixel ratio of the foreground region, the pixel ratio of the background region, and the overall average grayscale of the component image.
[0115] Step S104d2: Calculate the inter-class difference measure between the foreground region and the background region based on the average gray level of the foreground region, the average gray level of the background region, the pixel ratio of the foreground region, the pixel ratio of the background region, and the overall average gray level.
[0116] Inter-class difference measure refers to the measurement of grayscale difference between the foreground and background regions, which can be understood as the degree of distinction between the foreground and background after segmentation. The larger the inter-class difference measure, the better the segmentation effect.
[0117] In one embodiment, the process of calculating the inter-class difference metric between the foreground region and the background region includes: calculating the inter-class difference metric based on the average gray level of the foreground region, the average gray level of the background region, the pixel ratio of the foreground region, the pixel ratio of the background region, and the overall average gray level. The inter-class difference metric is calculated as: Foreground region pixel ratio × (Foreground region average gray level - Overall average gray level)² + Background region pixel ratio × (Background region average gray level - Overall average gray level)².
[0118] Step S104d3: Traverse the range of values for the segmentation threshold, and determine the segmentation threshold corresponding to the maximum value of the inter-class difference metric as the optimal segmentation threshold.
[0119] The optimal segmentation threshold refers to the segmentation threshold that maximizes the inter-class difference metric. This optimal segmentation threshold is used for subsequent binarization segmentation.
[0120] In one embodiment, the process of traversing the range of segmentation thresholds and determining the segmentation threshold corresponding to the maximum value of the inter-class difference metric as the optimal segmentation threshold includes: setting the range of segmentation thresholds; calculating the inter-class difference metric for each segmentation threshold; and determining the segmentation threshold corresponding to the maximum value of the inter-class difference metric as the optimal segmentation threshold.
[0121] Step S104d4: Perform binarization segmentation on the component image according to the optimal segmentation threshold, divide the pixel grayscale into two categories, obtain a binarized component image, and extract the component contour according to the binarized component image to perform pose calculation, and obtain the angle deviation and the position deviation.
[0122] The binarization segmentation refers to dividing pixel grayscale into two categories based on an optimal segmentation threshold. The binarized component image refers to the component image after binarization segmentation. The component outline refers to the boundary of the component in the binarized image.
[0123] In one embodiment, the process of binarizing the component image according to the optimal segmentation threshold includes: setting pixels with gray values greater than the optimal segmentation threshold as foreground values; setting pixels with gray values less than or equal to the optimal segmentation threshold as background values; obtaining a binarized component image; and extracting the component contour from the binarized component image for pose calculation.
[0124] In another embodiment, the step of calculating the pose of the component image according to the package type includes: performing preliminary edge detection on the component image to obtain edge pixel candidate points, and estimating sub-pixel coordinates based on the gray values of the edge pixel candidate points and their neighborhoods to find the non-integer coordinates with the largest edge response, thereby obtaining the component edge sub-pixel coordinates; the sub-pixel coordinate estimation adopts at least one of interpolation, fitting, or gray-scale moment calculation methods; when using interpolation, an interpolation function is selected based on the gray values of the edge pixel candidate points and their neighborhoods, linear interpolation is used when the gradient change between adjacent pixels satisfies the linear condition, and nonlinear interpolation is used when the edge curvature is greater than a preset curvature threshold; calculating the component angle and center offset based on the component edge sub-pixel coordinates, outputting the component angle as the angle deviation, outputting the center offset as the position deviation, and controlling the rotation axis to correct the deviation based on the angle deviation.
[0125] The preliminary edge detection refers to the process of performing edge detection on the component image to obtain candidate edge pixels. The sub-pixel coordinate estimation refers to the process of estimating sub-pixel coordinates based on the grayscale values of the candidate edge pixels and their neighborhoods. The interpolation method refers to the method of estimating sub-pixel coordinates using an interpolation function. The fitting method refers to the method of estimating sub-pixel coordinates by fitting an edge model. The grayscale moment calculation method refers to the method of estimating sub-pixel coordinates by calculating the grayscale moment. The linear interpolation refers to the interpolation method used when the gradient change between adjacent pixels satisfies a linear condition. The nonlinear interpolation refers to the interpolation method used when the edge curvature is greater than a preset curvature threshold.
[0126] In one embodiment, the process of performing preliminary edge detection and estimating sub-pixel coordinates on a component image includes: performing preliminary edge detection on the component image to obtain edge pixel candidate points; estimating sub-pixel coordinates based on the gray values of the edge pixel candidate points and their neighborhoods; when using interpolation, selecting an interpolation function based on the gray values of the edge pixel candidate points and their neighborhoods, using linear interpolation when the gradient change between adjacent pixels satisfies the linear condition, and using nonlinear interpolation when the edge curvature is greater than a preset curvature threshold; and calculating the component angle and center offset based on the component edge sub-pixel coordinates.
[0127] Specifically, if the gradient change in the region where the candidate edge pixels are located is gradual, linear interpolation is used to estimate the sub-pixel coordinates; if the edge curvature is greater than a preset curvature threshold, nonlinear interpolation is used to estimate the sub-pixel coordinates. By combining binarization segmentation and sub-pixel coordinate estimation, the accuracy of component contour extraction and pose calculation is improved.
[0128] More specifically, when the edge curvature is greater than a preset curvature threshold, nonlinear interpolation is used, including: Based on the gray values of the edge pixel candidate points and their neighborhood, a number of sampling pixels are selected along the edge gradient direction. An interpolation function is constructed using the coordinates of the sampled pixel as interpolation nodes, and the value of the interpolation function at the interpolation node is consistent with the gray value of the sampled pixel. The interpolation function is used to find the extreme values between the interpolation nodes, and the coordinates corresponding to the extreme values are determined as the sub-pixel coordinates of the element edge; When the extreme values of the interpolation function are not unique among the interpolation nodes, the extreme point with the smallest distance from the candidate edge pixel is selected as the sub-pixel coordinate of the element edge.
[0129] In this embodiment of the invention, a nonlinear interpolation method is adopted when the edge curvature is greater than a preset curvature threshold. This method involves selecting sampling pixels along the edge gradient direction, constructing an interpolation function, and determining the sub-pixel coordinates by finding the extreme values. This solves the problem of insufficient positioning accuracy of linear interpolation in the prior art when the edge curvature is large. It can improve the sub-pixel positioning accuracy at curved edges. Furthermore, when there are multiple extreme values in the interpolation function, a unique solution is determined by the distance criterion, which improves the robustness of sub-pixel coordinate calculation.
[0130] Existing linear interpolation typically selects sampling points along the coordinate axis direction (horizontal or vertical). When the edge is tilted or curved, the sampling direction does not coincide with the edge normal, resulting in inaccurate positions of the interpolation nodes across the edge and systematic deviations in sub-pixel coordinates.
[0131] The solution of this invention uses linear interpolation at low curvature edges, which is computationally efficient and fast; and nonlinear interpolation at high curvature edges, which is highly accurate and adaptable; and adaptive switching between the two interpolation methods is achieved through curvature threshold, balancing efficiency and accuracy; and the edge positioning accuracy of arc-shaped components (such as circular markers and BGA solder balls) is significantly improved.
[0132] In another embodiment, the step of performing pose calculation on the component image according to the package type includes: Preliminary edge detection is performed on the component image to obtain candidate edge pixels; Construct a gradient template covering multiple directions, calculate the gradient response of the candidate edge pixels along the multiple directions respectively, compare the gradient responses of each direction and determine the direction with the largest gradient response; Three adjacent pixel coordinate points are selected along the direction with the largest gradient response. An interpolation function is constructed using the three adjacent pixel coordinate points. The value of the interpolation function at the three adjacent pixel coordinate points is consistent with the gray value of the corresponding pixel. The interpolation function is used to find the extreme value between the three adjacent pixel coordinate points, and the coordinates corresponding to the extreme value are determined as the sub-pixel coordinates of the element edge; The element angle and center offset are calculated based on the sub-pixel coordinates of the element edge. The element angle is output as the angle deviation, and the center offset is output as the position deviation.
[0133] In this embodiment of the invention, by constructing a gradient template covering multiple directions and calculating the gradient response of candidate edge pixels along multiple directions, the problem of positioning deviation caused by the maximum gradient response direction being limited to the coordinate axis direction and the actual edge normal not coinciding with the sampling direction in the prior art is solved. By selecting three adjacent pixel coordinate points in the direction of the maximum gradient response to construct an interpolation function and finding the extreme value to determine the sub-pixel coordinates, the sub-pixel positioning accuracy of edges in any direction can be improved, thereby improving the calculation accuracy of element angle and center offset and reducing the secondary correction rate.
[0134] More specifically, a gradient template covering multiple directions is constructed, and the gradient response of the candidate edge pixels is calculated along each of the multiple directions. The gradient responses in each direction are compared, and the direction with the largest gradient response is determined, including: Construct gradient templates covering the horizontal, vertical, and at least two diagonal directions; Centered on the edge pixel candidate point, the gradient templates in each direction are used to perform convolution operations with the gray values of the edge pixel candidate point and its neighborhood to obtain the gradient response values in each direction. Compare the gradient response values in each direction, and determine the direction with the largest gradient response value as the direction with the largest gradient response; When the difference between the gradient response values in two directions is less than a preset difference threshold, the middle direction between the two directions is determined as the direction with the largest gradient response, and the gradient response value is recalculated along the middle direction.
[0135] The solution of this invention solves the problem of misjudgment of tilted edge direction caused by insufficient gradient template direction in the prior art by constructing a gradient template covering the horizontal direction, vertical direction and at least two diagonal directions and calculating the gradient response value in each direction by convolution operation; by selecting the middle direction and recalculating the gradient response value when the difference between the gradient response values in two directions is less than a preset difference threshold, the resolution of the direction with the maximum gradient response is further improved, the edge direction quantization error caused by template direction discretization is avoided, and the sub-pixel positioning accuracy and component pose calculation accuracy are improved.
[0136] Example 2 Please see Figure 2 , Figure 2 This is a schematic diagram of the automated control system for a pick-and-place machine, from calibration to placement, as disclosed in an embodiment of the present invention. Figure 2 As shown, the automated control system for this pick-and-place machine, from calibration to placement, may include: Acquisition module 21: used to acquire the calibration parameters and mounting configuration parameters of the pick and place machine. The calibration parameters include track calibration parameters, top-view camera calibration parameters, bottom-view camera calibration parameters and flight photography calibration parameters. The mounting configuration parameters include axis parameters, camera parameters, input and output parameters and process timing parameters. Acquisition module 22: After receiving the mounting start command, it controls the board feeding mechanism to transport the board to be mounted to the main platform and lock it, acquires the image of the marker point on the board to be mounted and calculates the coordinates of the marker point, and reads the component data table to obtain the package type, mounting coordinates and loading position of each component. Material picking module 23: It is used to control the placement head to move to the material feeding position to pick up the material in sequence according to the coordinates of the marked point, the calibration parameters and the placement configuration parameters, and then move to the flight photography position after picking up the material; Correction module 24: used to trigger the down-view camera to acquire component images at the flight photography position, perform pose calculation on the component images according to the package type to obtain angular deviation and position deviation, control the rotation axis to perform correction according to the angular deviation, and control the placement head to place the component on the board to be placed according to the corrected pose and the placement coordinates.
[0137] Example 3 Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of the present invention. The electronic device can be a computer, a server, etc. Of course, in certain cases, it can also be a mobile phone, tablet computer, monitoring terminal, or other smart device, as well as an image acquisition device with processing capabilities. Figure 3 As shown, the electronic device may include: Memory 510 storing executable program code; Processor 520 coupled to memory 510; The processor 520 calls the executable program code stored in the memory 510 to execute some or all of the steps in the automated control method of the pick-and-place machine from calibration to placement in Embodiment 1.
[0138] This invention discloses a computer-readable storage medium storing a computer program that causes a computer to perform some or all of the steps in the automated control method for a pick-and-place machine from calibration to placement as described in Embodiment 1.
[0139] This invention also discloses a computer program product, wherein when the computer program product is run on a computer, the computer executes some or all of the steps in the automated control method for the placement machine from calibration to placement in Embodiment 1.
[0140] This invention also discloses an application publishing platform, which is used to publish computer program products. When the computer program products are run on a computer, the computer executes some or all of the steps in the automated control method for the placement machine from calibration to placement in Embodiment 1.
[0141] In various embodiments of the present invention, it should be understood that the sequence number of each process does not necessarily imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0142] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they can be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0143] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0144] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several requests to cause a computer device (which can be a personal computer, server, or network device, specifically a processor in the computer device) to execute some or all of the steps of the methods described in the various embodiments of the present invention.
[0145] In the embodiments provided by this invention, it should be understood that B corresponding to A means that B is associated with A, and B can be determined based on A. However, it should also be understood that determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information.
[0146] Those skilled in the art will understand that some or all of the steps in the various methods of the embodiments described can be implemented by a program instructing related hardware. This program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.
[0147] The above provides a detailed description of the automated control method, system, electronic equipment, and storage medium for a pick-and-place machine from calibration to placement, as disclosed in the embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A method for automated control of a patcher from calibration to placement, characterized by, include: Obtain the calibration parameters and mounting configuration parameters of the pick-and-place machine. The calibration parameters include track calibration parameters, top-view camera calibration parameters, bottom-view camera calibration parameters, and flight photography calibration parameters. The mounting configuration parameters include axis parameters, camera parameters, input / output parameters, and process timing parameters. After receiving the mounting start command, the control board feeding mechanism will transport the board to be mounted to the main platform and lock it. The image of the marker point on the board to be mounted will be acquired and the coordinates of the marker point will be calculated. The component data table will be read to obtain the package type, mounting coordinates and loading position of each component. Based on the coordinates of the marked points, the calibration parameters, and the mounting configuration parameters, the patch head is sequentially controlled to move to the loading position to pick up the material, and then moves to the flight photography position after picking up the material. The downward-facing camera is triggered at the flight photography position to acquire component images. The pose of the component images is calculated according to the package type to obtain angular deviation and position deviation. The rotation axis is controlled to correct the deviation according to the angular deviation. The placement head is controlled to place the component onto the board to be placed according to the corrected pose and the placement coordinates.
2. The automated control method for a pick-and-place machine from calibration to placement as described in claim 1, characterized in that, The process of obtaining the calibration parameters of the pick-and-place machine includes: Control the width adjustment axis to return to zero and move it to the position of the movable track. Obtain the position coordinates of the movable track and the fixed track respectively. Calculate the track width based on the position coordinates of the movable track and the fixed track, and calibrate the position of the baffle. Control the upward-looking camera to acquire images at the calibrated position, calculate the pixel equivalent and camera angle based on the positioning coordinates of the two photos taken before and after moving a preset distance, and calibrate the offset of each patch head relative to the upward-looking camera. The downward-facing camera is controlled to trigger the taking of pictures and locate the center of the component. After aligning the center of the component with the center of the downward-facing camera, the first alignment position is recorded. The patch head is controlled to rotate by a preset angle and then aligned again and the second alignment position is recorded. The offset of each patch head relative to the downward-facing camera is calculated based on the first alignment position and the second alignment position. The Y-axis position, left X position, right X position, and the flight photography position of each patch head are calibrated respectively, and the calibration results are written into the configuration file.
3. The automated control method for a pick-and-place machine from calibration to placement as described in claim 2, characterized in that, The calculation of pixel equivalents and camera angles based on the positioning coordinates of two photos taken before and after moving a preset distance includes: Images are acquired at the calibrated location and marker points are located to obtain the first positioning coordinates; After controlling the motion mechanism to move a preset distance along a set direction, the image is acquired again and the same marker point is located to obtain the second positioning coordinates; The pixel equivalent is calculated based on the ratio of the pixel distance between the first positioning coordinate and the second positioning coordinate to the preset distance; The camera angle is calculated based on the coordinate difference between the first positioning coordinate and the second positioning coordinate, and the pixel equivalent and the camera angle are saved to the upward camera calibration parameters.
4. The automated control method for a pick-and-place machine from calibration to placement as described in claim 1, characterized in that, The step of performing pose calculation on the component image according to the package type includes: When the package type is the first type of package, the component image is smoothed, corner detection and interference point filtering are performed in sequence. Sub-pixel coordinates are calculated based on the filtered corners, and the component angle and center offset are calculated based on the sub-pixel coordinates. When the package type is the second type of package, the component image is smoothed and thresholded in sequence, the outline of each solder ball is extracted and the sub-pixel coordinates are calculated, the key solder balls are selected after circle fitting of each solder ball, and the component angle and center offset are calculated based on the center of the key solder balls. The calculated component angle is output as the angle deviation. The step of controlling the rotating shaft to correct the deviation based on the angle includes: When the package type is the first type of package, the number of pulses is calculated based on the angle deviation and the stepper motor is driven to rotate the corresponding number of steps to complete the correction. When the package type is the second type of package, the angle deviation is used as the position setpoint value and input to the position adjustment ring. The speed adjustment ring and the current adjustment ring are sequentially used to generate a switch control signal to drive the brushless motor to rotate and complete the correction. Record the actual angle after correction and compare it with the angle deviation. If the comparison result exceeds the preset range, it is determined that a second correction is required.
5. The automated control method for a pick-and-place machine from calibration to placement as described in claim 1, characterized in that, The step of triggering the downward-looking camera to acquire images at the flight photography location includes: The patch head is controlled to move to the flight photography position according to the flight photography calibration parameters; Trigger the downward-facing camera to take a picture and wait for the material handling visual time; The visual pose calculation is invoked to obtain the angle deviation, and the R-axis rotation is controlled to correct the deviation based on the angle deviation. The angle deviation and correction results are written into the operation log, and the log is recorded and the current element is skipped or the photo is retaken when the visual calculation fails.
6. The automated control method for a pick-and-place machine from calibration to placement as described in claim 1, characterized in that, The step of triggering the image acquisition element of the downward-looking camera at the flight photography position includes: when the suction head enters the flight photography position, the motion control card directly triggers the flight camera to take a picture according to the encoder position. When the suction head enters the flight photography window, the motion control card directly triggers the flight camera to take a picture based on the encoder position, including: During the movement of the suction head, the motion control card reads the position of the suction head encoder in real time; Compare the encoder position with the flight photography position; When the encoder position enters the flight photography position, the motion control card generates a trigger signal and sends it to the flight camera and the stroboscopic light source; after receiving the trigger signal, the flight camera exposes in a global shutter mode, and the stroboscopic light source illuminates for a duration of microseconds during the exposure to obtain an image of the flying material.
7. The automated control method for a pick-and-place machine from calibration to placement as described in claim 6, characterized in that, After obtaining the image of the flying material, the process also includes: During the movement of the suction head, the encoder collects the suction head position data in real time, and the inertial measurement unit collects the camera vibration data in real time. A point spread function is constructed based on the suction head position data and the camera vibration data. The point spread function characterizes the motion trajectory of the material on the image sensor during exposure. The image of the flying material is deconvolved using the point spread function to restore the material outline and obtain a clear image of the material; the deconvolution operation of the image of the flying material using the point spread function includes: The ambiguity index is calculated based on the point spread function. The fuzziness level index is compared with a preset fuzziness threshold; When the blur level index is greater than the preset blur threshold, deconvolution operation is performed on the flying material image; When the blur level index is less than or equal to the preset blur threshold, the deconvolution operation is skipped, and the flying material image is directly used as a clear material image.
8. The automated control method for a pick-and-place machine from calibration to placement as described in claim 1, characterized in that, After triggering the image acquisition element of the downward-looking camera at the stated flight photography location, the following is also included: An optimization framework constrained by gradient sparsity is constructed, in which gradient minimization is used to smooth the component image; the smoothing process is controlled by a smoothing parameter and a scaling parameter, wherein the smoothing parameter controls the degree of smoothing and the scaling parameter controls the edge preservation ratio. The smoothing parameter and the scaling parameter are adjusted according to the background grayscale of the nozzle, the size of the components, and the lighting conditions in the actual production environment. The smoothed component image is used as input for subsequent pose calculation.
9. The automated control method for a pick-and-place machine from calibration to placement as described in claim 1, characterized in that, The step of performing pose calculation on the component image according to the package type includes: The grayscale distribution of the component image is statistically analyzed, and a segmentation threshold is set to divide the pixels of the component image into a foreground region and a background region. The average grayscale of the foreground region, the average grayscale of the background region, the pixel ratio of the foreground region, the pixel ratio of the background region, and the overall average grayscale of the component image are calculated respectively. Based on the average grayscale of the foreground region, the average grayscale of the background region, the pixel ratio of the foreground region, the pixel ratio of the background region, and the overall average grayscale, calculate the inter-class difference measure between the foreground region and the background region; Traverse the range of values for the segmentation threshold, and determine the segmentation threshold corresponding to the maximum value of the inter-class difference metric as the optimal segmentation threshold; The component image is binarized according to the optimal segmentation threshold, and the pixel grayscale is divided into two categories to obtain a binarized component image. The component contour is extracted from the binarized component image to perform pose calculation, and the angle deviation and the position deviation are obtained. Alternatively, the step of performing pose calculation on the component image based on the package type includes: Preliminary edge detection is performed on the component image to obtain edge pixel candidate points. Sub-pixel coordinates are estimated based on the gray values of the edge pixel candidate points and their neighborhoods to find the non-integer coordinates with the largest edge response, thereby obtaining the component edge sub-pixel coordinates. The subpixel coordinate estimation employs at least one of interpolation, fitting, or gray-scale moment calculation methods. When interpolation is used, an interpolation function is selected based on the edge pixel candidate points and their neighborhood gray-scale values. Linear interpolation is used when the gradient change between adjacent pixels satisfies the linear condition, and nonlinear interpolation is used when the edge curvature is greater than a preset curvature threshold. The element angle and center offset are calculated based on the sub-pixel coordinates of the element edge. The element angle is output as the angle deviation, and the center offset is output as the position deviation. The rotation axis is then controlled to correct the deviation based on the angle deviation.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program causes a computer to perform the automated control method for a pick-and-place machine from calibration to placement as described in any one of claims 1 to 9.
