Camera target capturing method and device
Through the camera, multiple photos and image analysis of the target holes are solved, the problem of low target grabbing accuracy is improved, the positioning accuracy of circuit board processing is improved, and scrapping is reduced.
Patent Information
- Application Number
- CN202310517805.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-09
- Publication Date
- 2024-11-12
AI Technical Summary
It is difficult for existing cameras to accurately identify the target holes of the circuit board during the target grabbing process, resulting in processing and positioning errors and causing the circuit board to be scrapped.
The target hole was taken multiple times by the camera, multiple target hole images were obtained, and image feature extraction and coordinate comparison were used to judge the target grab accuracy and obtain actual coordinates, and adjust the photography parameters to improve recognition accuracy.
It effectively avoids misidentification caused by taking a photo and grabbing the target, improves the camera's target accuracy, and reduces the scrap rate of circuit boards.
Smart Images

Figure CN118945458A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to a camera target capturing method and device. Background Art
[0002] With the development of 5G communication technology, the requirements for the process accuracy of circuit boards (PCBs) are getting higher and higher. For example, when the wiring of circuit boards becomes more and more dense and the aperture becomes smaller and smaller, the requirements for the processing and positioning accuracy of circuit boards are also getting higher and higher. If there is a deviation in the positioning, the processed circuit boards will be scrapped.
[0003] At present, in order to improve the processing positioning accuracy of the circuit board, a target hole is usually set on the circuit board, and then a camera is used to take a picture of the target hole to obtain the position of the target hole, thereby realizing the position correction of the circuit board. However, after the circuit board has undergone multiple processing processes, the state around the target hole will change, such as warping, distortion, etc., which will cause the camera to be unable to capture the target hole or mistakenly identify other holes / spots as target holes. When the camera mistakenly identifies the target hole, the processing system cannot know this error and will still perform the circuit board position correction according to the identified target hole, and then process the circuit board according to the wrong position correction result, thereby causing the circuit board to be scrapped.
[0004] Therefore, how to improve the camera's target capture accuracy is an urgent problem to be solved. Summary of the invention
[0005] The purpose of the present invention is to provide a camera target capturing method and device to solve the problem of how to improve the camera's target capturing accuracy.
[0006] In order to solve the above technical problems, the present invention provides a camera target capturing method, the camera target capturing method comprising:
[0007] Control the camera to move to the target coordinates;
[0008] Using a camera to take pictures of the target hole multiple times to obtain multiple target hole images;
[0009] Performing target hole recognition on each of the target hole images, and obtaining image coordinates of the target hole recognized in each of the target hole images;
[0010] The plurality of image coordinates are used to determine whether the camera's target-grasping accuracy meets the requirements, and / or the plurality of image coordinates are used to obtain the actual coordinates of the target hole.
[0011] Optionally, in the camera target capturing method, when the camera is used to take pictures of the target hole multiple times, the shooting parameters of each picture are not exactly the same, and the shooting parameters include light source brightness, light source color, exposure time, focal length and white balance.
[0012] Optionally, in the camera target capturing method, when adjusting the photographing parameters, the adjustment range is ±10%.
[0013] Optionally, in the camera target capturing method, a method for performing target hole recognition on each target hole image includes:
[0014] Performing image feature extraction within a preset central range of the target hole image to identify holes within the preset central range of the target hole image;
[0015] Calculating the pore diameter of the pore;
[0016] The aperture is compared with the theoretical aperture of the target aperture, and if the difference between the two is within the aperture difference threshold range, the aperture is marked as the target aperture.
[0017] Optionally, in the camera target capturing method, the method of performing target hole recognition on each target hole image further includes:
[0018] If the image feature extraction identifies at least two holes, and the difference between the apertures of the at least two holes and the theoretical aperture of the target hole is within the aperture difference threshold range, the hole with the smallest difference is marked as the target hole.
[0019] Optionally, in the camera target capturing method, a method of using a plurality of the image coordinates to determine whether the camera target capturing accuracy meets the requirements, and / or a method of using a plurality of the image coordinates to obtain the actual coordinates of the target hole includes:
[0020] Comparing the plurality of image coordinates in pairs to obtain a plurality of image coordinate deviations;
[0021] Determine whether the image coordinate deviation exceeds the deviation threshold. If all the image coordinate deviations do not exceed the deviation threshold, then: determine that the camera's target capture accuracy meets the requirements, and / or perform fitting processing on all the image coordinates, and use the coordinates obtained after the fitting processing as the actual coordinates of the target hole.
[0022] Optionally, in the camera target capturing method, the deviation threshold includes a first deviation threshold and a second deviation threshold; the camera target capturing method includes:
[0023] Determine whether the image coordinate deviation exceeds a first deviation threshold, and if all the image coordinate deviations do not exceed the first deviation threshold, determine that the camera's target capture accuracy meets the requirements;
[0024] And / or, determine whether the image coordinate deviation exceeds a second deviation threshold; if all the image coordinate deviations do not exceed the second deviation threshold, perform fitting processing on all the image coordinates, and use the coordinates obtained after the fitting processing as the actual coordinates of the target hole.
[0025] Optionally, in the camera target capture method, the image coordinate deviation includes image X deviation, image Y deviation and image center deviation;
[0026] The method for determining whether the image coordinate deviation exceeds a first deviation threshold comprises: determining whether the image X deviation and the image Y deviation respectively exceed the first deviation threshold, and / or determining whether the image center deviation exceeds the first deviation threshold;
[0027] The method for determining whether the image coordinate deviation exceeds a second deviation threshold comprises: determining whether the image X deviation and the image Y deviation respectively exceed the second deviation threshold, and / or determining whether the image center deviation exceeds the second deviation threshold.
[0028] In order to solve the above technical problems, the present invention further provides a camera target capturing device, which is used to perform the camera target capturing method as described in any one of the above items, and the camera target capturing device comprises:
[0029] A workbench, used for placing an object to be tested, wherein the object to be tested has a target hole;
[0030] A camera, used to take a picture of the target hole to obtain a target hole image;
[0031] The control processor is used to obtain the target hole image and identify the target hole in the target hole image to obtain the image coordinates of the target hole, thereby using the image coordinates to determine whether the camera's target capture accuracy meets the requirements, and / or obtain the actual coordinates of the target hole.
[0032] Optionally, in the camera target capturing device, the control processor includes a drive control unit, an image acquisition unit, an image processing unit, an analysis and judgment unit, and a result output unit;
[0033] The driving control unit is used to control the camera to move to the target grabbing coordinates;
[0034] The image acquisition unit is used to receive the target hole image acquired by the camera;
[0035] The image processing unit is used to identify the target hole in the target hole image to obtain the image coordinates of the target hole;
[0036] The analysis and judgment unit is used to use the image coordinates to judge whether the camera's target capture accuracy meets the requirements, and / or to obtain the actual coordinates of the target hole;
[0037] The result output unit is used to output the result of whether the camera's target capture accuracy meets the requirements, and / or the actual coordinates of the target hole.
[0038] The present invention provides a method and device for capturing a target with a camera. The method comprises: controlling a camera to move to a target capturing coordinate; using a camera to take multiple photos of a target hole to obtain multiple images of the target hole; performing target hole recognition on each of the target hole images, and obtaining the image coordinates of the target hole recognized in each of the target hole images; using multiple image coordinates to determine whether the accuracy of the camera capturing the target meets the requirements, and / or using multiple image coordinates to obtain the actual coordinates of the target hole. By taking multiple photos of the target hole, analyzing and processing the target hole images obtained by the multiple photos to obtain the result of whether the accuracy of the camera capturing the target meets the requirements, and / or the actual coordinates of the target hole, the situation of capturing the wrong target hole due to capturing the target by taking a single photo is avoided, and the problem of how to improve the accuracy of the camera capturing the target is solved. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 A flow chart of the camera target capturing method provided in this embodiment;
[0040] Figure 2 A flowchart of step S3 provided for this embodiment;
[0041] Figure 3 A flowchart of a preferred step S3 provided in this embodiment;
[0042] Figure 4 A flowchart of step S4 provided for this embodiment;
[0043] Figure 5 A schematic diagram of the structure of the camera target grabbing device provided in this embodiment;
[0044] Figure 6 A schematic diagram of the structure of the control processor provided in this embodiment;
[0045] Figure 7 A schematic diagram of the principle of fitting image coordinates provided in this embodiment;
[0046] The symbols in the figure are explained as follows:
[0047] 100 - workbench; 200 - camera; 300 - control processor; 310 - drive control unit; 320 - image acquisition unit; 330 - image processing unit; 340 - analysis and judgment unit; 350 - result output unit; 400 - object to be tested; 410 target hole; 411 - image target hole; 412 - fitting target hole. DETAILED DESCRIPTION
[0048] The camera target grabbing method and device proposed in the present invention are further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the accompanying drawings are all in a very simplified form and use non-precise proportions, which are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention. In addition, the structure shown in the accompanying drawings is often a part of the actual structure. In particular, the emphasis that each accompanying drawing needs to show is different, and sometimes different proportions are used.
[0049] It should be noted that the terms "first", "second", etc. in the specification, claims and drawings of the present invention are used to distinguish similar objects in order to describe the embodiments of the present invention, rather than to describe a specific order or sequence. It should be understood that the structures used in this way can be interchanged under appropriate circumstances. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0050] This embodiment provides a camera target capture method, such as Figure 1 As shown, the camera target capturing method includes:
[0051] S1, control the camera to move to the target coordinates;
[0052] S2, taking photos of the target hole multiple times using a camera to obtain multiple target hole images;
[0053] S3, performing target hole recognition on each of the target hole images, and obtaining image coordinates of the target hole recognized in each of the target hole images;
[0054] S4 uses the multiple image coordinates to determine whether the camera's target capture accuracy meets the requirements, and / or uses the multiple image coordinates to obtain the actual coordinates of the target hole.
[0055] Among them, S41 in step S4 uses multiple image coordinates to determine whether the camera's target capture accuracy meets the requirements and S42 uses multiple image coordinates to obtain the actual coordinates of the target hole. Either or both can be performed, depending on the specific usage scenario and needs.
[0056] The camera target grasping method provided in this embodiment takes multiple photos of the target hole, and analyzes and processes the target hole images obtained by the multiple photos to obtain the result of whether the camera's target grasping accuracy meets the requirements and / or the actual coordinates of the target hole, thereby avoiding the situation where the wrong target hole is grasped due to capturing the target with one photo, and solves the problem of how to improve the camera's target grasping accuracy.
[0057] The target grabbing coordinates are usually the theoretical coordinates of the target hole. Theoretically, when the camera reaches the target grabbing coordinates, the image of the target hole should be at the center of the camera's imaging surface. In actual target grabbing, since the machine will position and fix the circuit board, even if there are some equipment deviations, the actual coordinates of the target hole will not deviate too much from the theoretical coordinates, so that a clear target hole image can be obtained through camera imaging. If the target grabbing coordinates are too far from the actual coordinates of the target hole, the target hole may not be photographed due to the limitation of the camera's shooting angle, or the target hole in the acquired target hole image may be distorted due to the inevitable distortion of the outer edge of the camera's optical system, which is not conducive to subsequent target hole recognition.
[0058] Considering that the circuit board processing process goes through lamination, electroplating, drilling, solder mask and other process steps, the state around the target hole of the circuit board will change, such as circuit board surface warping, target hole distortion, board surface ink infiltration into the target hole, burrs in the target hole, board surface reflection, etc., which will cause the camera to fail to capture the target or capture the target incorrectly. In order to improve the adaptability of the camera when taking pictures of different circuit board conditions, so that the target hole image can clearly and correctly reflect the target hole contour, in this embodiment, the camera will be used to take pictures of the target hole multiple times, and when the camera is used to take pictures of the target hole multiple times, the shooting parameters of each picture are not exactly the same, and the shooting parameters include light source brightness, light source color, exposure time, focal length and white balance.
[0059] In order to take into account both the accuracy and efficiency of target capture, the target hole is usually photographed 3 to 8 times to obtain 3 to 8 target hole images for analysis and processing. For example, the target hole is photographed 6 times, and the light source brightness, light source color, exposure time, focal length and white balance are adjusted each time according to the control variable method. For example, the first shooting is performed according to the initially set light source brightness, light source color, exposure time, focal length and white balance parameters; the second shooting keeps other parameters unchanged, the light source brightness is increased, and the shooting is performed again; the third shooting restores the light source brightness to the initial state, changes the light source color, and keeps other parameters unchanged, and the shooting is performed again... In this way, 6 target hole images with different imaging effects can be obtained, and each target hole image should reflect the state of the target hole differently. In this way, by analyzing the 6 target hole images with different imaging effects (target hole identification), the occurrence of target capture errors caused by taking a single shot can be avoided to a certain extent.
[0060] In addition to adjusting the photographing parameters individually according to the control variable method, multiple photographing parameters can also be adjusted each time. When adjusting multiple photographing parameters each time, multiple sets of photographing parameters can be provided in advance based on photographing experience, so that they can be directly called when adjusting the photographing parameters. Multiple sets of photographing parameters should be in line with the current photographing environment, so as to ensure that a clear target hole image can be captured.
[0061] Of course, a better approach is that the adjustment of the photographing parameters is carried out under adaptation or deep learning, that is, the photographing parameters of the next photographing are automatically adjusted according to the effect of the target hole image obtained by the previous photographing, so as to obtain photographing parameters that better highlight the characteristics of the target hole. Under this method, the adjustment of each photographing parameter is not limited to a single parameter, and multiple parameters can be adjusted synchronously to achieve a better photographing effect. However, considering that this method will perform a large number of background calculations, resulting in a long processing time, this approach is not recommended in the field of online target capture, but the approach previously described in this embodiment is adopted, that is, each adjustment is mechanically adjusted according to the preset adjustment value, but what kind of photographing parameters are adjusted each time the photographing is taken can be pre-set by the program, or can be randomly specified by the program, and a single parameter can be adjusted, or multiple parameters can be adjusted. The specific adjustment method of the photographing parameters and the program implementation method can be set by those skilled in the art according to actual needs, which is common knowledge in the field and will not be repeated here.
[0062] Preferably, in this embodiment, if the program does not store multiple photographing parameters in advance, but needs to adjust the photographing parameters online, when adjusting the photographing parameters, the adjustment range is controlled within ±10%. For example, if the initial exposure time is 0.008s, the adjustment range is between 0.0072s and 0.0088s, for example, adjusted to 0.0075s.
[0063] Since the initially set shooting parameters are usually the best shooting parameters obtained through a lot of shooting experience, it is only necessary to fine-tune each parameter when taking multiple photos. If the difference in the adjustment of the shooting parameters is too large, it may cause overexposure, insufficient lighting, and out of focus, resulting in the inability to recognize each feature of the acquired target hole image, making the target hole image unusable.
[0064] Further, in step S3, if Figure 2 As shown, the method for performing target hole recognition on each target hole image includes:
[0065] S31, performing image feature extraction within a preset central range of the target hole image to identify holes within the preset central range of the target hole image;
[0066] S32, calculating the pore diameter of the pore;
[0067] S33, comparing the aperture with the theoretical aperture of the target aperture, and if the difference between the two is within an aperture difference threshold range, marking the aperture as a target aperture.
[0068] In step S31, image feature extraction is performed only on pixels within a preset range in the center of the target hole image. Firstly, the amount of calculation for image feature extraction on the target hole image can be reduced, thereby improving the processing speed of target hole recognition. Secondly, interference factors that affect target hole recognition, such as holes / spots appearing on the periphery of the target hole image, can be avoided as much as possible.
[0069] The image feature extraction technology used in this embodiment is the HOG feature algorithm. Of course, in other embodiments, other feature extraction algorithms may also be used. Using feature extraction technology to identify target holes is a technology well known to those skilled in the art and will not be described in detail here.
[0070] The theoretical aperture of the target hole is pre-stored in the program. Specifically, it can be obtained from the drilling program or manually entered. After the aperture of the hole extracted by features is obtained, it is compared with the pre-stored theoretical aperture of the target hole. If the difference between the two is within the aperture difference threshold, the hole extracted by features can be identified as the target hole. The aperture difference threshold can be set according to actual conditions. It should be noted that when obtaining the aperture difference, the aperture of the hole in the target hole image needs to be proportionally converted according to the ratio of the target hole image to the actual object to obtain the actual aperture of the hole in the target hole image, and then the actual aperture of the hole identified by feature extraction is compared with the theoretical aperture of the target hole.
[0071] Since the apertures of various holes in the circuit board are relatively small, usually between 0.5 and 3.7 mm, and can even reach 0.1 mm if laser drilling is used, the aperture difference threshold should not be set too large to ensure the accuracy of target hole identification. In this embodiment, the aperture difference threshold is set within 5% of the theoretical aperture of the target hole, that is,
[0072] Diff 阈 <0.05×D 靶孔
[0073] Among them, Diff 阈 is the aperture difference threshold, D 靶孔 is the theoretical aperture of the target hole.
[0074] For example, when the theoretical aperture of the target hole is 3mm, the aperture difference threshold can be set to 0.1mm (less than 0.15mm (0.05×3mm)). If the aperture of the identified hole is 2.96mm, the difference between the two is 0.04mm (3mm-2.96mm), which is less than the aperture difference threshold of 0.1mm, then the hole is marked as a target hole; if the aperture of the identified hole is 2.82mm, the difference between the two is 0.18mm (3mm-2.82mm), which is greater than the aperture difference threshold of 0.1mm, then the hole is not a target hole.
[0075] Furthermore, in this embodiment, if Figure 3 As shown, the method of performing target hole recognition on each target hole image in step S3 further includes:
[0076] S34, if the image feature extraction identifies at least two holes, and the difference between the apertures of the at least two holes and the theoretical aperture of the target hole is within the aperture difference threshold range, then mark the hole with the smallest difference as the target hole.
[0077] Taking into account possible deviations in feature extraction, holes / spots that are not target holes are identified as holes, and the difference between the hole diameters obtained after feature extraction of these holes / spots and the theoretical hole diameter of the target hole is also within the aperture difference threshold range. In order to ensure that the correct target hole can be identified with a greater probability in the presence of interfering holes, the hole with the smallest difference is marked as the target hole.
[0078] Still taking the above example (the theoretical aperture of the target hole is 3mm, and the aperture difference threshold is set to 0.1mm), if the image feature extraction identifies three holes at this time, with apertures of 2.23mm, 2.91mm and 3.03mm respectively, the hole corresponding to the aperture of 2.23mm is excluded because the difference between the hole corresponding to the aperture of 2.91mm and the hole corresponding to the aperture of 3.03mm and the theoretical aperture of the target hole exceeds the aperture difference threshold; but the difference between the hole corresponding to the aperture of 2.91mm and the hole corresponding to the aperture of 3.03mm and the theoretical aperture of the target hole are both less than the aperture difference threshold (0.09mm and 0.03mm), but the difference between the hole corresponding to the aperture of 3.03mm and the theoretical aperture of the target hole is smaller (0.03mm<0.09mm), so the hole corresponding to the aperture of 3.03mm is marked as the target hole.
[0079] Of course, there is a very small but more complicated situation: multiple holes are identified, and the difference between the apertures of at least two holes and the theoretical aperture of the target hole is within the aperture difference threshold range, and the difference between the apertures of at least two holes and the theoretical aperture of the target hole is consistent. For example, a hole with an aperture of 2.97 mm and a hole with an aperture of 3.03 mm are identified, or two holes with apertures of 3.03 mm are identified. Which hole should be selected as the target hole?
[0080] This embodiment provides a solution: select the hole closer to the center point of the target hole image as the target hole. This is because when the camera is initially used to shoot, the camera's target coordinates fall at the theoretical coordinates of the target hole, that is, the center point of the target hole image corresponds to the theoretical coordinates of the target hole, so the accuracy of selecting the hole closer to the center point of the target hole image as the target hole is higher.
[0081] In this embodiment, if Figure 4 As shown, step S4 uses the multiple image coordinates to determine whether the camera's target capture accuracy meets the requirements, and / or the method of using the multiple image coordinates to obtain the actual coordinates of the target hole includes:
[0082] S401, comparing the plurality of image coordinates in pairs to obtain a plurality of image coordinate deviations;
[0083] S402, determining whether the image coordinate deviation exceeds a deviation threshold. If all the image coordinate deviations do not exceed the deviation threshold, determining that the camera's target capture accuracy meets the requirements, and / or performing fitting processing on all the image coordinates, and using the coordinates obtained after the fitting processing as the actual coordinates of the target hole.
[0084] Wherein, in step S401, the image coordinate deviation includes image X deviation, image Y deviation and image center deviation. Image X deviation is the difference between the X-axis values of the two image coordinates, image Y deviation is the difference between the Y-axis values of the two image coordinates, and image center deviation is the difference between the center points of the two image coordinates. If the first image coordinate is (X1, Y1) and the second image coordinate is (X2, Y2), then image X deviation ΔX = X1-X2, image Y deviation ΔY = Y1-Y2, image center deviation
[0085] In step S402, the deviation threshold includes a first deviation threshold and a second deviation threshold. It is determined whether the image coordinate deviation exceeds the first deviation threshold. If all the image coordinate deviations do not exceed the first deviation threshold, it is determined that the camera's target capture accuracy meets the requirements; and / or it is determined whether the image coordinate deviation exceeds the second deviation threshold. If all the image coordinate deviations do not exceed the second deviation threshold, all the image coordinates are fitted, and the coordinates obtained after the fitting process are used as the actual coordinates of the target hole.
[0086] Specifically, in this embodiment, the method for judging whether the image coordinate deviation exceeds the first deviation threshold includes: judging whether the image X deviation and the image Y deviation exceed the first deviation threshold respectively, and / or judging whether the image center deviation exceeds the first deviation threshold. In the specific application process, it can be judged whether only the image X deviation and the image Y deviation exceed the first deviation threshold respectively, or it can be judged whether only the image center deviation exceeds the first deviation threshold, or the two judgments can be combined. For example, if neither the image X deviation nor the image Y deviation exceeds the first deviation threshold, it is judged that the camera target capture accuracy meets the requirements; or if the image center deviation does not exceed the first deviation threshold, it is judged that the camera target capture accuracy meets the requirements. Generally speaking, one of the two judgment methods can be selected, but in order to ensure that there is no error in the calculation, both judgment methods can be selected at the same time, but this will increase the amount of calculation and reduce the processing speed.
[0087] If all the image coordinate deviations do not exceed the first deviation threshold, it is determined that the camera's target capture accuracy meets the requirements. If there is an image coordinate deviation that exceeds the first deviation threshold, it is determined that the camera's target capture accuracy does not meet the requirements. At this time, the machine can be stopped and an alarm can be issued to prompt the staff to intervene and confirm the target hole status to avoid processing abnormalities or scrap due to inaccurate target capture.
[0088] The first deviation threshold is preset manually. After theoretical calculation and a large amount of empirical data, it is better to set the first deviation threshold at 2-5μm. This range can adapt to the camera target capture of most circuit boards on the market, and can ensure that the accuracy of the target capture is accurately judged while taking into account efficiency.
[0089] Similarly, in this embodiment, the method for determining whether the image coordinate deviation exceeds the second deviation threshold includes: determining whether the image X deviation and the image Y deviation exceed the second deviation threshold respectively, and / or determining whether the image center deviation exceeds the second deviation threshold.
[0090] In this embodiment, the actual coordinates of the target hole are obtained by fitting all the image coordinates. Specifically, when the image coordinate deviation does not exceed the second deviation threshold, it means that the coordinates of the holes identified by the multiple target hole images are not much different from each other, and the actual coordinates of the target hole can be obtained by image fitting. There are many ways to fit. For example, a circle encirclement method is used, that is, a circle is used to include all the hole coordinates, and the diameter of the circle is the smallest, then the center coordinates of the circle are the actual coordinates of the target hole. Another example is the mean calculation method, which is a simpler method, by taking the average value of all image coordinates to obtain the actual coordinates of the target hole.
[0091] When there is an image coordinate deviation exceeding the second deviation threshold, the machine can be shut down and an alarm can be sounded to prompt the staff that the actual coordinates of the target hole cannot be obtained. Alternatively, a group of image coordinates exceeding the second deviation threshold can be marked, and the image coordinate deviation with other image coordinates can be confirmed, thereby filtering out the data of holes with large deviations, and then the image coordinates of the remaining holes are used for image fitting to obtain the actual coordinates of the target hole.
[0092] Similar to the first deviation threshold, the second deviation threshold is also preset manually. After theoretical calculation and a large amount of empirical data, it is better to set the second deviation threshold within 5μm. Within this range, the deviation between the actual coordinates of the target hole obtained by image fitting and the actual situation can be guaranteed to be within an acceptable range. In actual application, the second deviation threshold can be the same as the first deviation threshold or different.
[0093] This embodiment also provides a camera target grabbing device, such as Figure 5 As shown, including:
[0094] The workbench 100 is used to place the object to be tested 400, and the object to be tested 400 has a target hole 410;
[0095] The camera 200 is used to take a picture of the target hole 410 to obtain a target hole image;
[0096] The control processor 300 is used to obtain the target hole image and identify the target hole in the target hole image to obtain the image coordinates of the target hole, thereby using the image coordinates to determine whether the camera's target capture accuracy meets the requirements, and / or obtain the actual coordinates of the target hole.
[0097] In this embodiment, the workbench 100 carries and fixes the object to be tested 400 to ensure that the object to be tested 400 does not move during the photographing process; the camera 200 takes multiple photographs of the target hole 410 of the object to be tested 400 to obtain the target hole image; the control processor 300 obtains the target hole image from the camera 200, identifies the target hole, obtains the image coordinates of the target hole, and finally determines whether the target capture accuracy of the camera 200 meets the requirements, and / or obtains the actual coordinates of the target hole. In this way, the situation of mis-capturing the target caused by the camera 200 capturing the target once can be avoided, and the target capture accuracy can be improved.
[0098] Furthermore, in this embodiment, if Figure 6 As shown, the control processor 300 includes a driving control unit 310, an image acquisition unit 320, an image processing unit 330, an analysis and judgment unit 340 and a result output unit 350; the driving control unit 310 is used to control the camera 200 to move to the target capture coordinates; the image acquisition unit 320 is used to receive the target hole image acquired by the camera 200; the image processing unit 330 is used to identify the target hole in the target hole image to obtain the image coordinates of the target hole; the analysis and judgment unit 340 is used to use the image coordinates to determine whether the camera's target capture accuracy meets the requirements, and / or, to obtain the actual coordinates of the target hole; the result output unit 350 is used to output the result of whether the camera's target capture accuracy meets the requirements, and / or, the actual coordinates of the target hole.
[0099] Specifically, the driving control unit 310 is electrically connected to the camera 200, and controls the movement of the camera 200 and takes pictures through the burned program. The image acquisition unit 320 is connected to the camera 200 in a wireless or wired manner, so as to acquire the target hole image taken by the camera 200. The image processing unit 330 is connected to the image acquisition unit 320 in a wireless or wired manner, so as to receive the target hole image, and the image processing unit 330 is equipped with functional modules such as a feature extraction architecture and a coordinate calculation architecture, which are used to process the target hole image to obtain the image coordinates of the target hole. The analysis and judgment unit 340 is connected to the image processing unit 330 in a wireless manner to receive the image coordinates provided by the image processing unit 330, and the analysis and judgment unit 340 is equipped with a calculation module, which is used to calculate the image coordinate deviation, and compare it with the deviation threshold, so as to determine whether the camera's target capture accuracy meets the requirements, and the analysis and judgment unit 340 is also equipped with a fitting processing module, which is used to fit all the image coordinates to obtain the actual coordinates of the target hole. The result output unit 350 is in communication connection with the analysis and judgment unit 340 , and is used to receive the result generated by the analysis and judgment unit 340 (the result of judging whether the accuracy of the camera's target capture meets the requirements, and / or the actual coordinates of the target hole).
[0100] Preferably, when receiving the target hole image, the image acquisition unit 320 can mark the target hole image according to the shooting process of the camera 200. For example, when taking pictures of the same circuit board 5 times, the identification code of the circuit board can be marked on the target hole image, so that the target hole image can correspond to the actual circuit board one by one, and the shooting order of the 5 target hole images can be marked in sequence, so as to facilitate the acquisition of information such as shooting parameters. In addition, other information can be marked according to actual needs.
[0101] Typically, the result output unit 350 is connected to a human-computer interaction interface in communication, so that the obtained data can be displayed visually for easy reference by the staff. Of course, the result output unit 350 can also be connected to a cloud database in communication, so that the obtained data can be stored in the cloud database. In addition, the result output unit 350 can also be connected to the drive control unit 310, so that instructions are issued to the drive control unit 310 based on the obtained data to control the operation of the camera 200. For example, when the data obtained by the result output unit 350 includes that the camera's target capture accuracy does not meet the requirements, the result output unit 350 sends an instruction to the drive control unit 310, so that the drive control unit 310 controls the camera 200 to stop operating and issues an alarm to prompt the staff to intervene to confirm the situation.
[0102] The following is a specific example of how the camera target capturing device performs camera target capturing.
[0103] In this embodiment, a PCB drilling machine with a camera is selected as the processing equipment for illustration, and the camera adopts an industrial camera CCD; the size of the PCB board is 25 inches × 30 inches, and a target hole with a theoretical aperture of 3.2 mm is set in the upper left corner of the PCB board.
[0104] It should be noted that, in specific applications, the camera target grabbing method provided by this application is not limited to being applied to drilling machines; the camera target grabbing device is not limited to being configured on drilling machines, and can be equipment such as milling machines, panel splitters, and even AOI optical inspection machines; the camera is not limited to industrial camera CCDs, but can also be other types of imaging devices; the target to be tested is not limited to PCB boards, but can be any target to be tested. It should also be noted that when applied to other targets, the camera target grabbing method and device solutions that can be obtained by technicians in this field without a doubt based on the content of this application should also fall within the scope of protection of this application.
[0105] refer to Figure 5 As shown, the workbench 100 is used to place a PCB board 400 with a target hole 410, and the camera 200 can be set next to the main shaft of the drilling machine (not shown in the figure), so that it can move in the XYZ direction to reach the target grabbing coordinate. The camera 200 is connected to the control processor 300 for communication, so as to achieve communication between the two.
[0106] Before the camera takes a picture of the target hole, the camera's target coordinates can be preset in the drilling machine. In this embodiment, the target coordinates are (15, 15). In this way, when the PCB board 400 is positioned on the workbench 100, the theoretical position of the target hole is at the target coordinates (15, 15). In fact, due to equipment tolerance, the target hole will actually fall near the target coordinates, that is, it can be captured by the camera and located in the center area of the captured target hole image. Then the drilling machine is initialized to ensure the accuracy of the camera's moving position. In addition, the camera's shooting parameters can also be adjusted according to the current situation of the PCB board.
[0107] After the preparation work is completed, the PCB board 400 is placed on the workbench 100 and fixed. Then the camera 200 is controlled to move to the target grabbing coordinates, and at this time, the target hole 410 on the PCB board 400 falls within the shooting field of view of the camera 200.
[0108] Next, the target hole 410 is photographed multiple times using the camera 200 to obtain multiple target hole images. In this embodiment, the target hole 410 is photographed 6 times, and the photographing parameters for each time are as shown in the following table:
[0109] No. Light source brightness Light source color Exposure time focal length White Balance 1 450lm White 1 / 120s 285mm 5300 2 450lm White 1 / 120s 270mm 5400 3 440lm White 1 / 120s 285mm 5400 4 460lm red 1 / 120s 285mm 5700 5 470lm green 1 / 110s 285mm 5200 6 490lm blue 1 / 100s 285mm 4800
[0110] Thus, six target hole images with different imaging effects were obtained.
[0111] Next, the control processor 300 receives the six target hole images, performs target hole recognition on each target hole image, and obtains the image coordinates of the target hole recognized in each target hole image.
[0112] Specifically, the image acquisition unit 320 in the control processor 300 receives 6 target hole images and records them in sequence as p2301001.a, p2301001.b, p2301001.c, p2301001.d, p2301001.e and p2301001.f, where p2301001 is the identification code of the PCB board, and a, b, c, d, e, and f correspond to 6 photos respectively.
[0113] Next, the image processing unit 330 performs feature extraction on the target hole image to identify the target hole, and obtains the image coordinates of the target hole after identifying the target hole. Specifically, the image processing unit 330 first selects the image within the 30mm×30mm area extending from the center point of the target hole image as the image source for feature extraction and analysis, which can save a lot of computing time and improve the accuracy of target capture. Of course, in other embodiments, the selected area range can be set according to the repeatability of the machine, processing error, etc.; then the image processing unit 330 performs image feature extraction on the image source to identify the holes in the image source; then, after calculation processing, the apertures of all identified holes can be obtained. Among them, the aperture of the hole identified in p2301001.a is 3.18mm; two holes are identified in p2301001.b, and the apertures of the two holes are 2.20mm and 3.24mm respectively; the aperture of the hole identified in p2301001.c is 3.21mm; three holes are identified in p2301001.d, and the apertures of the three holes are 3.16mm, 3.23mm and 3.65mm respectively; the aperture of the hole identified in p2301001.e is 3.20mm; three holes are identified in p2301001.f, and the apertures of the three holes are 3.12mm, 3.17mm and 3.23mm respectively.
[0114] It should be noted that the feature extraction of the image and the calculation of the aperture of the identified hole are well known to those skilled in the art and will not be described in detail here. In addition, in this embodiment, in order to facilitate the staff to check the processing process of the control processor 300, the control processor 300 can save all the target hole images and mark the selected image source range and the appearance and aperture of the identified hole in the target hole image.
[0115] After obtaining the aperture of the identified hole, the image processing unit 330 will compare it with the theoretical aperture of the target hole (3.2mm), and judge it according to the pre-stored aperture difference threshold, so as to mark each target hole image as a target hole. In this embodiment, the aperture difference threshold is set to 0.05mm. Specifically, in p2301001.a, the difference between the aperture of the identified hole and the theoretical aperture of the target hole is 0.02mm
[0116] (3.18mm-3.2mm), which does not exceed the aperture difference threshold of 0.05mm, so this hole is marked as a target hole; similarly, in p2301001.b, since the difference between the hole with an aperture of 2.20mm and the theoretical aperture of the target hole exceeds the aperture difference threshold, the hole is not marked as a target hole, while the difference between the hole with an aperture of 3.24mm and the theoretical aperture of the target hole does not exceed the aperture difference threshold, so this hole is marked as a target hole; in p2301001.c, the difference between the aperture of the identified hole and the theoretical aperture of the target hole does not exceed the aperture difference threshold, so this hole is marked as a target hole; in p2301001.d, since the differences between the holes with an aperture of 3.16mm and an aperture of 3.23mm and the theoretical aperture of the target hole do not exceed the aperture difference threshold, but the hole with an aperture of 3.23mm The difference between the theoretical aperture of the target hole and the hole with an aperture of 3.65 mm is smaller, so the hole with an aperture of 3.23 mm is marked as a target hole. The difference between the theoretical aperture of the target hole and the hole with an aperture of 3.65 mm exceeds the aperture difference threshold, so the hole is not marked as a target hole. In p2301001.e, the difference between the aperture of the identified hole and the theoretical aperture of the target hole does not exceed the aperture difference threshold, so the hole is marked as a target hole. In p2301001.f, since the difference between the theoretical aperture of the target hole and the hole with an aperture of 3.12 mm exceeds the aperture difference threshold, the hole is not marked as a target hole. The differences between the theoretical aperture of the target hole and the hole with an aperture of 3.17 mm and the hole with an aperture of 3.23 mm do not exceed the aperture difference threshold, and the differences between the two holes and the theoretical aperture of the target hole are consistent, so both holes are marked as target holes (abnormal target holes) for further confirmation.
[0117] After marking the target hole, the image processing unit 330 further analyzes and calculates to obtain the image coordinates of the marked target hole. Specifically, the image coordinates of the target hole identified in p2301001.a are (15.003, 14.997); the image coordinates of the target hole identified in p2301001.b are (15.002, 14.995); the image coordinates of the target hole identified in p2301001.c are (15.003, 14.997); the image coordinates of the target hole identified in p2301001.d are (14.998, 14.996); The image coordinates of the target hole identified in p2301001.e are (15.001, 15.000); the image coordinates of the two target holes identified in p2301001.f are (23.006, 15.083) and (14.999, 14.998), respectively. Here, since the target hole position of (14.999, 14.998) is closer to the center point (15, 15) of the target hole image, the hole is confirmed as the target hole identified by the p2301001.f target hole image.
[0118] Then, the analysis and judgment unit 340 uses the image coordinates to judge whether the camera's target capture accuracy meets the requirements, and obtains the actual coordinates of the target hole. In this embodiment, the analysis and judgment unit 340 needs to complete two tasks: first, judging the camera's target capture accuracy; second, obtaining the actual coordinates of the target hole. Because it is necessary to determine whether the camera's target capture accuracy meets the requirements before obtaining the actual coordinates of the target hole, the two tasks in this embodiment have a sequence: first, judging the camera's target capture accuracy, and then obtaining the actual coordinates of the target hole on the premise that the camera's target capture accuracy meets the requirements.
[0119] Specifically, the analysis and judgment unit 340 pre-stores a first deviation threshold for judging the accuracy of the camera's target capture and a second deviation threshold for obtaining the actual coordinates of the target hole. In this embodiment, the first deviation threshold and the second deviation threshold are both 5 μm.
[0120] The analysis and judgment unit 340 compares the image coordinates of the target holes in the six target hole images in pairs to obtain the image coordinate deviation. In this embodiment, only the image center deviation is used. The following table shows the results of the analysis and judgment unit 340 calculating the target hole coordinates in the six target hole images using the image center deviation:
[0121]
[0122]
[0123] It can be seen from the above table that the image coordinate deviations between the image coordinates of the target holes obtained in the 6 target hole images of this embodiment do not exceed the first deviation threshold. Therefore, the analysis and judgment unit 340 determines that the camera target capture accuracy meets the requirements.
[0124] Next, the analysis and judgment unit 340 confirms that the deviations of these image coordinates do not exceed the second deviation threshold, so all the image coordinates are fitted and the coordinates obtained after the fitting process are used as the actual coordinates of the target hole. Figure 7 The analysis and judgment unit 340 overlaps the six target hole images, so that the target holes (image target holes) 411 marked in the six target hole images fall into the same layer 400; then a circle with the smallest diameter is used to surround all the six image target holes 411, and the center of the circle is the fitted target hole 412, and the coordinates of the center of the circle are the actual coordinates of the target hole, which are (15.001, 14.997).
[0125] Finally, the result output unit outputs two pieces of data: "The camera's target-grabbing accuracy meets the requirements" and "The actual coordinates of the target hole are (15.001, 14.997)". After that, the PCB board is replaced and the next round of operation begins.
[0126] The camera target grabbing method and device provided in this embodiment include: controlling the camera to move to the target grabbing coordinates; taking multiple pictures of the target hole with the camera to obtain multiple target hole images; identifying the target hole for each of the target hole images, and obtaining the image coordinates of the target hole identified in each of the target hole images; using the multiple image coordinates to determine whether the camera target grabbing accuracy meets the requirements, and / or using the multiple image coordinates to obtain the actual coordinates of the target hole. By taking multiple pictures of the target hole, analyzing and processing the target hole images obtained by the multiple pictures to obtain the result of whether the camera target grabbing accuracy meets the requirements, and / or the actual coordinates of the target hole, the situation of grabbing the wrong target hole due to taking a single picture to grab the target is avoided, and the problem of how to improve the camera target grabbing accuracy is solved.
[0127] The above description is only a description of the preferred embodiments of the present invention, and is not intended to limit the scope of the present invention. Any changes or modifications made by a person skilled in the art in the field of the present invention based on the above disclosure shall fall within the scope of protection of the claims.
Claims
1. A camera target capturing method, characterized in that: The camera target capturing method comprises: Control the camera to move to the target coordinates; Using a camera to take pictures of the target hole multiple times to obtain multiple target hole images; Performing target hole recognition on each of the target hole images, and obtaining image coordinates of the target hole recognized in each of the target hole images; The plurality of image coordinates are used to determine whether the camera's target-grasping accuracy meets the requirements, and / or the plurality of image coordinates are used to obtain the actual coordinates of the target hole.
2. The camera target capturing method according to claim 1, characterized in that: When the camera is used to take photos of the target hole for multiple times, the photography parameters of each photo are not completely the same, and the photography parameters include light source brightness, light source color, exposure time, focal length and white balance.
3. The camera target capturing method according to claim 2, characterized in that: When adjusting the photographing parameters, the adjustment range is ±10%.
4. The camera target capturing method according to claim 1, characterized in that: The method for performing target hole recognition on each target hole image comprises: Performing image feature extraction within a preset central range of the target hole image to identify holes within the preset central range of the target hole image; Calculating the pore diameter of the pore; The aperture is compared with the theoretical aperture of the target aperture, and if the difference between the two is within the aperture difference threshold range, the aperture is marked as the target aperture.
5. The camera target capturing method according to claim 4, characterized in that: The method for performing target hole recognition on each of the target hole images further includes: If the image feature extraction identifies at least two holes, and the difference between the apertures of the at least two holes and the theoretical aperture of the target hole is within the aperture difference threshold range, the hole with the smallest difference is marked as the target hole.
6. The camera target capturing method according to claim 1, characterized in that: The method of using the plurality of image coordinates to judge whether the accuracy of the camera's target capture meets the requirements, and / or using the plurality of image coordinates to obtain the actual coordinates of the target hole includes: Comparing the plurality of image coordinates in pairs to obtain a plurality of image coordinate deviations; Determine whether the image coordinate deviation exceeds the deviation threshold. If all the image coordinate deviations do not exceed the deviation threshold, then: determine that the camera's target capture accuracy meets the requirements, and / or perform fitting processing on all the image coordinates, and use the coordinates obtained after the fitting processing as the actual coordinates of the target hole.
7. The camera target capturing method according to claim 6, characterized in that: The deviation threshold includes a first deviation threshold and a second deviation threshold; the camera target capture method includes: Determine whether the image coordinate deviation exceeds a first deviation threshold, and if all the image coordinate deviations do not exceed the first deviation threshold, determine that the camera's target capture accuracy meets the requirements; And / or, determine whether the image coordinate deviation exceeds a second deviation threshold; if all the image coordinate deviations do not exceed the second deviation threshold, perform fitting processing on all the image coordinates, and use the coordinates obtained after the fitting processing as the actual coordinates of the target hole.
8. The camera target capturing method according to claim 7, characterized in that: The image coordinate deviation includes image X deviation, image Y deviation and image center deviation; The method for determining whether the image coordinate deviation exceeds a first deviation threshold comprises: determining whether the image X deviation and the image Y deviation respectively exceed the first deviation threshold, and / or determining whether the image center deviation exceeds the first deviation threshold; The method for determining whether the image coordinate deviation exceeds a second deviation threshold comprises: determining whether the image X deviation and the image Y deviation respectively exceed the second deviation threshold, and / or determining whether the image center deviation exceeds the second deviation threshold.
9. A camera target capturing device, used for performing the camera target capturing method according to any one of claims 1 to 8, characterized in that: The camera target grabbing device comprises: A workbench, used for placing an object to be tested, wherein the object to be tested has a target hole; A camera, used to take a picture of the target hole to obtain a target hole image; The control processor is used to obtain the target hole image and identify the target hole in the target hole image to obtain the image coordinates of the target hole, thereby using the image coordinates to determine whether the camera's target capture accuracy meets the requirements, and / or obtain the actual coordinates of the target hole.
10. The camera target grabbing device according to claim 9, characterized in that: The control processor includes a drive control unit, an image acquisition unit, an image processing unit, an analysis and judgment unit, and a result output unit; The driving control unit is used to control the camera to move to the target grabbing coordinates; The image acquisition unit is used to receive the target hole image acquired by the camera; The image processing unit is used to identify the target hole in the target hole image to obtain the image coordinates of the target hole; The analysis and judgment unit is used to use the image coordinates to judge whether the camera's target capture accuracy meets the requirements, and / or to obtain the actual coordinates of the target hole; The result output unit is used to output the result of whether the camera's target capture accuracy meets the requirements, and / or the actual coordinates of the target hole.
