Multi-angle rotation 3D guide dispensing and detection method based on two-dimensional vision

Abstract

The invention relates to the field of dispensing. The method comprises the following steps: setting a rotation angle of a camera, selecting a 3D calibration path and angle, pre-processing a 3D calibration image, calibrating a 2D camera VS and 3D camera, setting 2D Mark, setting a scanning path of the 3D camera, pre-processing a 3D dispensing image, exporting a dispensing template path, setting dispensing parameters, and pre-processing detection after dispensing; according to the invention, the processing of multi-angle 3D visual images is added, and the precise positioning of the 2D camera on the product Mark is combined, so that the dynamic compensation of the multi-angle 3D camera on the dispensing path of various complicated structure products is realized, the problems of poor consistency of the customer product assembly process, single scanning path of the 3D camera and non-horizontal dispensing plane are solved, the actual dispensing path is completely precise and controllable, and the production efficiency is improved. Dependence of product process assembly tolerance is greatly reduced, multi-point height detection CT is saved, more complex products are compatible, and 2D or 3D dispensing coordinates can be freely selected.

Description

Technical Field

[0001] The present invention belongs to the field of glue dispensing, and in particular relates to patent names. Background Art

[0002] At present, in the field of dispensing, products are becoming more and more sophisticated, and processes are becoming more and more complex. In order to meet customer needs, developing new dispensing programming methods and coping with more complex products is a direction that more and more manufacturers must optimize. The existing dispensing programming method is to use the Mark on the picture captured by the 2D camera as the positioning point, and then use the 2D camera on the sample to edit the instructions at the location where dispensing is required, and insert the height detection instructions where there are height changes. This programming method has the following disadvantages: when facing products that need to be dispensed in multiple areas and multiple locations, the programming efficiency is low; the process layout precision between the product's Mark and the dispensing position is high; for products with frequent height changes and no fixed rules on the dispensing plane, there is no way to accurately compensate for the dispensing height; a fixed-angle 3D camera is installed on the three-axis dispensing equipment, and due to the principle of triangular imaging, it is easy to have dead angles; due to product differences and dispensing height problems, the detection effect after dispensing with a 2D camera is poor. Therefore, in order to simultaneously meet the process of the above scenarios, the existing technology has defects and needs to be improved. Summary of the invention

[0003] The present invention makes up for the shortcomings of 2D and 3D visual processing, combines the advantages of 2D and 3D, and solves the technical problems existing in the production of prior art.

[0004] To this end, the present invention proposes a multi-angle rotation 3D guidance and detection method based on two-dimensional vision, which highlights the accuracy of 2D positioning Mark, combines the high positioning accuracy of 3D camera, has spatial guidance capability, and enhances the detection accuracy by integrating guidance and detection, and compensates for the limitations of 3D camera triangular imaging with rotation. This solution has the advantages of high programming efficiency, accurate capture of coordinates, automatic trajectory fitting, deviation protection, and free combination of 2D and 3D programming.

[0005] To solve the above problems, the technical solution provided by the present invention is as follows: a method for multi-angle rotation 3D guided dispensing detection based on two-dimensional vision, characterized in that the steps are as follows:

[0006] 3D camera rotation angle setting: calibrate the angle deviation caused by mechanical assembly

[0007] Select 3D calibration path and angle: select different angles and scanning directions to calibrate 2D and 3D cameras;

[0008] 3D calibration image preprocessing: According to the set angle and direction, the obtained 3D image is converted into a grayscale image for preprocessing;

[0009] 2D camera VS 3D camera calibration: calibrate the coordinates of the 3D image preprocessing with the 2D camera coordinates;

[0010] 2D Mark Setting: Recording mark points for overall offset and angle compensation of 3D scanning path;

[0011] 3D camera scanning path setting: set the 2D Mark reference coordinates to determine the 3D scanning path and angle of the template;

[0012] 3D dispensing image preprocessing: preprocess the grayscale image obtained by the 3D camera;

[0013] Export dispensing template path: select the path that needs to be dispensed as the template program reference point;

[0014] Dispensing parameter setting: set the dispensing speed, dispensing amount, offset and dispensing height of the parameter type;

[0015] Post-dispensing detection preprocessing: Detect the dispensing effect by comparing the 3D grayscale images before and after dispensing.

[0016] Preferably; before implementing the rotation angle setting of the 3D camera, it is also necessary to enable 3D image inversion and set 3D camera guide deviation protection. Enabling 3D image inversion makes it convenient for customers to view 3D pictures; setting 3D camera guide deviation protection: according to the benchmark Mark point and benchmark mechanical dispensing coordinates of the set template, the theoretical mechanical coordinates are deduced through the Mark coordinates and angles of different products; based on the difference between the actual glue path of the 3D camera grabbing the Mark of different product marks and the theoretical mechanical coordinates, protection is performed. If the protection value is exceeded, a fitting or shutdown alarm will be issued to prevent 3D camera grabbing errors.

[0017] Preferably; the rotation angle setting of the 3D camera includes a preset posture angle and a fine-tuning posture angle; the preset posture angles include 0°, 45° and 90°, and the rotation angle of the 3D camera is selected according to demand; the fine-tuning posture angle is to fine-tune the preset posture angle.

[0018] Preferably; select the 3D calibration path and angle: select the rotation angle and scanning direction of the 3D camera. By selecting the scanning start and end point of the 3D camera, the scanning range of the 3D camera includes the entire calibration block.

[0019] Preferred; 3D calibration image preprocessing: use a 3D camera device to obtain the three-dimensional contour information of the calibration block, convert the calibration block height data map into a grayscale image, and preprocess the grayscale image; perform preprocessing on the grayscale image, capture the falling edge of the side wall of the 9-point circular hole on the calibration block, and fit the center coordinates of the 9-point circular hole; after the preprocessing is completed, rescan the calibration block to obtain the absolute coordinates of the 9-point circular hole relative to the scanning path.

[0020] Preferred; 2D camera VS 3D camera calibration: use a 2D camera to teach the 2D coordinates of the center of each circular hole, use circle matching to find the center positions of 9 circular holes, and correspond them one by one with the coordinates of the 3D camera in sequence; use a 9-point calibration algorithm, establish a linear equation group based on the above 2D and 3D coordinates, and solve the calibration relationship of the 2D camera VS 3D camera.

[0021] Preferred; 2D Mark Setting: Use a 2D camera to locate the mark and establish a coordinate system.

[0022] Optimized; 3D camera scanning path setting: select line laser guidance in the sub-template to enter the line laser guidance editing interface; all parameter settings in the line laser guidance setting are saved with the sub-template, and the sub-template can be placed in the main template for offset and matrix placement. You can choose to guide the dispensing inspection of the main template program one by one or perform guidance → dispensing → inspection for all main templates; in the online laser guidance editing interface, set the length, speed, scanning angle and scanning direction of the scanning path; select the guidance number and the inspection number, which represent the pre-processing storage location of the grayscale image converted from the three-dimensional height data scanned by the 3D camera. After the path is generated, the template Mark is used as the reference, and the actual Mark of the product will be used to compensate for the deviation of the scanning path during actual production.

[0023] Preferred; 3D dispensing image preprocessing: grayscale image is processed to display product outline; select the detection area on the product; through pixel arrangement, pixel fitting and format conversion similar processing methods, the mouse selects the selected area and a cross-sectional height enlarged image based on the shape of the selected detection area will appear on the right;

[0024] By detecting the spatial height with a 3D camera, you can select the maximum value, minimum value, first peak, and first trough related information in the capture interface as the reference point, and output the reference point or the deviation value based on the reference point, plane falling edge and other parameters.

[0025] Preferably; export the dispensing template path: after the 3D camera preprocesses the grayscale image, it is converted into coordinates in the 2D coordinate system. According to the dispensing coordinates, the instruction type of the instruction is selected to add the program, and the coordinates added in the program and the corresponding instruction type are set as the benchmark as the dispensing coordinate benchmark for the deviation protection calculation.

[0026] Compared with the prior art, the beneficial effect of the present invention is that, by adopting the above scheme, the processing of multi-angle 3D visual images is newly added, and combined with the precise positioning of the product Mark by the 2D camera, dynamic compensation of the dispensing path of various complex structure products by the multi-angle 3D camera is realized, and the problems of poor consistency of customer product assembly process, single 3D camera scanning path and uneven dispensing plane are solved, so that the actual dispensing path is completely accurately controllable, the dependence of product process assembly tolerance is greatly reduced, multi-point height detection CT is saved, it is compatible with more complex products and 2D or 3D dispensing coordinates can be freely selected. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0028] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0029] In order to facilitate the understanding of the present invention, the present invention is described in more detail below in conjunction with the accompanying drawings and specific embodiments. Preferred embodiments of the present invention are provided in the accompanying drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described in this specification. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive.

[0030] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "fixed", "integrally formed", "left", "right" and similar expressions used in this specification are for illustrative purposes only. In the drawings, units with similar structures are marked with the same reference numerals.

[0031] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in this specification in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0032] like Figure 1 As shown, one embodiment of the present invention is: a method for multi-angle rotation 3D guided dispensing detection based on two-dimensional vision, the steps are as follows:

[0033] 3D camera rotation angle setting: calibrate the angle deviation caused by mechanical assembly. Select 3D calibration path and angle: select different angles and scanning directions to calibrate 2D and 3D cameras.

[0034] 3D calibration image preprocessing: According to the set angle and direction, the obtained 3D image is converted into a grayscale image for preprocessing;

[0035] 2D camera VS 3D camera calibration: calibrate the coordinates of the 3D image preprocessing with the 2D camera coordinates;

[0036] 2D Mark setting: record the mark points for overall offset and angle compensation of the 3D scanning path; 3D camera scanning path setting: set the 2D Mark reference coordinates to determine the 3D scanning path and angle of the template;

[0037] 3D dispensing image preprocessing: preprocess the grayscale image obtained by the 3D camera;

[0038] Export dispensing template path: select the path that needs to be dispensed as the template program reference point; dispensing parameter setting: set the dispensing speed, glue output, offset and dispensing height corresponding to the parameter type;

[0039] Pre-processing after dispensing: Detect the dispensing effect by comparing the 3D grayscale images before and after dispensing. Before implementing the rotation angle setting of the 3D camera, it is also necessary to enable 3D image inversion and set the 3D camera guide deviation protection. Enabling 3D image inversion makes it easier for customers to view 3D images; setting 3D camera guide deviation protection: according to the reference mark points and reference mechanical dispensing coordinates of the set template, the theoretical mechanical coordinates are inferred through the mark coordinates and angles of different products; based on the difference between the actual operation glue path captured by the 3D camera of different product marks and the theoretical mechanical coordinates, protection is performed. If the protection value is exceeded, a fitting or shutdown alarm will be issued to prevent 3D camera capture errors.

[0040] It should be noted that: 3D image inversion: This function is designed to invert the color or contrast of a 3D image to make it easier to observe and analyze. This is particularly useful for dealing with dark or highly reflective objects, as the original image may have difficult to see the details. By inverting, important features can be highlighted for easier recognition and understanding by the user.

[0041] 3D camera guided deviation protection: This is a safety mechanism that aims to prevent misoperation caused by the image captured by the camera not being consistent with the expected one. Specifically, the system compares the difference between the actual captured mark and the pre-defined standard position. If the difference exceeds the predetermined threshold, the system will trigger an alarm to alert the operator that there may be potential risks, or directly stop the relevant operation to prevent accidents.

[0042] 3D image inversion and 3D camera guided deviation protection help users fully utilize the capabilities of 3D cameras to perform complex vision tasks while ensuring the safety and reliability of the entire process.

[0043] The rotation angle setting of the 3D camera includes preset attitude angle and fine-tuning attitude angle; the preset attitude angles include 0°, 45° and 90°, and the rotation angle of the 3D camera is selected according to needs; the fine-tuning attitude angle is to fine-tune the preset attitude angle.

[0044] It should be noted that during the calibration process, the 3D camera needs to be placed at a specific angle to ensure accurate 3D data. This is because the 3D camera builds a 3D model by measuring the light reflected from the surface of the object, and this process is very sensitive to the relative position between the camera and the object being measured. If the position or angle of the camera is incorrect, the measurement results may be distorted or inaccurate. Therefore, in some cases, setting the correct rotation angle is crucial to successfully completing the calibration;

[0045] The preset attitude angles include 0°, 45° and 90°. The rotation angle of the 3D camera can be selected according to the needs. The following functions can be achieved through special specific angles:

[0046] Viewing angle coverage: 3D cameras have an optimal operating range, within which they can obtain the clearest and most accurate data. Adjusting to a specific angle can help the camera better cover the entire area of ​​interest.

[0047] Reduce occlusion: When there are complex structures in the scene (for example, multiple objects occlude each other), adjusting the angle appropriately can minimize the impact of these occlusions on the final image.

[0048] Improve accuracy: The amount and quality of information captured by the camera may vary at different angles. Choosing the right angle can help improve measurement accuracy.

[0049] By making rational use of the angle setting function, not only can the challenges brought by mechanical assembly be effectively overcome, but the application flexibility and accuracy of 3D cameras can also be significantly improved.

[0050] Furthermore, the rotation angle setting of the 3D camera also includes control options, including the adjustment of parameters such as angle, voltage, current, etc. These parameters will affect the output intensity and stability of the laser, and thus affect the quality of 3D imaging.

[0051] Select 3D calibration path and angle: Select the rotation angle and scanning direction of the 3D camera. By selecting the scanning start and end points of the 3D camera, the scanning range of the 3D camera includes the entire calibration block.

[0052] It should be noted that the rotation angle here refers to the preset posture angle and the fine-tuning posture angle; scanning action direction: the user selects according to the three directions of X, Y, and Z, and the 3D camera starts scanning according to the direction selected by the user, thereby covering different sides of the calibration block.

[0053] Scan start and end point: The user enters the start and end position coordinates of the 3D camera scan. By setting these coordinates accurately, you can ensure that the camera's scanning range just covers the entire calibration block, without wasting too much resources or missing any important details.

[0054] Based on the above points, by carefully selecting the rotation angle, scanning direction, and start and end points of the 3D camera, the user can ensure that the camera's scanning trajectory can completely cover all surfaces of the calibration block. The purpose of this is not only to collect enough data to build a complete 3D model, but also to avoid modeling defects or errors caused by insufficient scanning range.

[0055] 3D calibration image preprocessing: Use a 3D camera device to obtain the three-dimensional contour information of the calibration block, convert the calibration block height data map into a grayscale image, and preprocess the grayscale image; perform preprocessing on the grayscale image, capture the falling edge of the side wall of the 9-point circular hole on the calibration block, and fit the center coordinates of the 9-point circular hole; after the preprocessing is completed, rescan the calibration block to obtain the absolute coordinates of the 9-point circular hole relative to the scanning path.

[0056] It should be noted that: the 3D contour information of the calibration block is obtained by using a 3D camera device and converted into a grayscale image for preprocessing, and then the center coordinates of the 9 circular holes on the calibration block are extracted, and finally the absolute coordinates of these circular holes relative to the scanning path are obtained. We can explain it in detail according to the following steps:

[0057] Step 1: Obtain 3D contour information and convert it into grayscale image

[0058] Preparation phase: First, we need to prepare the 3D camera equipment and calibration block. The calibration block is a special geometric shape with feature points of known size on it, which is used to calibrate the measurement accuracy of the camera.

[0059] Collect data: Use a 3D camera device to scan the calibration block and obtain its 3D contour information. This usually involves moving and rotating the camera to ensure that all areas of interest are covered.

[0060] Data conversion: Convert the collected 3D contour information into a grayscale image. This step is usually completed through a projection algorithm, that is, mapping points in 3D space to a 2D plane to form a grayscale image. The brightness of the pixel in the grayscale image represents the distance of the point from the camera.

[0061] Step 2: Grayscale image preprocessing

[0062] Noise removal: Filter the grayscale image to eliminate noise interference. Common filtering methods include mean filtering, median filtering, etc.

[0063] Edge enhancement: Use gradient operators (such as Sobel operator and Canny operator) to enhance the edge information in the image to facilitate subsequent feature extraction.

[0064] Threshold segmentation: By performing threshold segmentation on the grayscale image, the image is divided into foreground (calibrated block) and background (non-calibrated block area), simplifying the subsequent processing steps.

[0065] Step 3: Extract the coordinates of the center of the circular hole

[0066] Feature point detection: On the preprocessed grayscale image, use Hough transform or other circle detection algorithms to find the approximate locations of the 9 circular holes on the calibration block.

[0067] Refine the center position: For each detected circular hole, use the least square method or other curve fitting method to refine the calculation of the center coordinates of the circular hole. This step requires the use of the falling edge information of the circular hole, that is, the location of the grayscale mutation at the boundary of the circular hole.

[0068] Step 4: Get the absolute coordinates

[0069] Rescan the calibration block: Scan the calibration block again, this time to get the absolute coordinates of each hole relative to the scan path. This means we need to know the exact position of each hole in 3D space.

[0070] Coordinate transformation: transform the coordinates of the center of each circular hole from the image coordinate system to the world coordinate system. This usually involves the calibration of the camera's internal and external parameters, as well as the transformation matrix calculation between coordinate systems.

[0071] Record the results: Finally, we will get a set of coordinates of the center points of 9 circular holes. These coordinates are absolute coordinates with reference to the scanning path.

[0072] Through the above steps, we can not only calibrate the measurement accuracy of the 3D camera, but also provide basic geometric information for subsequent computer vision tasks (such as object recognition, posture estimation, etc.).

[0073] 2D camera VS 3D camera calibration: Use a 2D camera to teach the 2D coordinates of the center of each circular hole, use circle matching to find the center positions of the 9 circular holes, and correspond them one by one with the coordinates of the 3D camera in sequence; use a 9-point calibration algorithm, establish a linear equation group based on the above 2D and 3D coordinates, and solve the calibration relationship of the 2D camera VS 3D camera.

[0074] It should be noted that: to use the 2D camera to teach the 2D coordinates of the center of each circular hole, the circle matching method is used to find the center positions of the 9 circular holes, which correspond to the coordinates of the 3D camera in sequence, and a 9-point calibration algorithm is used. According to the above 2D and 3D coordinates, a linear equation system is established to solve the calibration relationship between the 2D camera and the 3D camera. We can explain it in detail according to the following steps:

[0075] Step 1: Use a 2D camera to teach the 2D coordinates of the center of each circular hole

[0076] Preparation stage: First, we need to prepare the 2D camera and the calibration plate. The calibration plate has circular holes of known size and distribution, which will be used as reference points in subsequent steps.

[0077] Capture image: Use a 2D camera to capture an image of the calibration plate. Make sure the camera is stable to avoid shaking and blurring.

[0078] Image processing: Preprocess the captured images, including denoising, contrast enhancement, etc., to facilitate subsequent feature extraction.

[0079] Hole location: Use image processing techniques (such as Hough transform, edge detection, etc.) to find all the holes in the image. The centers of these holes are the 2D coordinates we are looking for.

[0080] Step 2: Use circle matching to find the center positions of the 9 circular holes

[0081] Feature matching: Based on the first step, the circle matching method is used to confirm which circular holes are the 9 we are interested in. This usually involves calculating the diameter and other geometric properties of each circular hole, and then comparing them with the design specifications of the calibration plate.

[0082] Sorting and numbering: Assign unique identifiers to the nine holes for ease of subsequent processing. These identifiers should reflect the actual order of the holes on the calibration plate.

[0083] Step 3: Match the coordinates of the 3D camera one by one in order

[0084] 3D coordinate acquisition: Use a 3D camera to obtain the depth information of the same calibration plate. This will give the position of each circular hole in three-dimensional space.

[0085] Pairing: Pair the center coordinates of the circular hole obtained by the 2D camera with the position of the circular hole obtained by the 3D camera. This step depends on the identifier assigned in the second step.

[0086] Step 4: Use the 9-point calibration algorithm

[0087] Establish a set of equations: Based on the paired 2D and 3D coordinates, establish a set of linear equations. This set of equations describes the mathematical relationship between the 2D camera and the 3D camera.

[0088] Solve the system of equations: Use appropriate mathematical methods (such as least squares) to solve this system of equations to obtain the calibration relationship between the 2D camera and the 3D camera. This is usually expressed as a transformation matrix that can convert 2D coordinates to corresponding 3D coordinates.

[0089] Through the above steps, we not only obtain the independent coordinate information of the 2D camera and the 3D camera, but also establish a calibration relationship between the two. This calibration relationship is very important for many computer vision and robotics applications because it allows us to seamlessly switch and fuse between two different types of data sources.

[0090] 2D Mark Setting: Use a 2D camera to locate the mark and establish a coordinate system.

[0091] It should be noted that: Advantages of 2D cameras

[0092] High accuracy

[0093] The product mark itself has high precision: the existing production technology is very mature and can ensure the production precision and accuracy of the mark.

[0094] 2D cameras have high capture accuracy: Compared with 3D cameras, 2D cameras have higher accuracy in capturing planar marks.

[0095] Diverse processing modes Template matching: Applicable to situations where the mark pattern is known, and rapid positioning is achieved by comparing it with the template in the database.

[0096] Circle matching: Suitable for circular mark recognition, and determines the position by detecting the center and radius. Edge matching: Suitable for marks with clear edges, and locates by detecting edge features. Body recognition: Recognizes the shape of the mark as a whole, and is suitable for marks with complex shapes. Edge matching: Combines edge and internal features for matching to improve the robustness of recognition.

[0097] Rich production experience and wide application: 2D cameras are widely used in the field of industrial automation and have accumulated a lot of production and debugging experience.

[0098] 2. Limitations of 3D Cameras

[0099] Unable to effectively capture flat marks: 3D cameras mainly rely on depth information. When facing flat marks, there is a lack of effective depth changes, which makes recognition difficult.

[0100] Utilizing the advantages of 2D cameras: In view of the many advantages of 2D cameras in Mark positioning, the present invention selects 2D cameras as the main Mark capturing method.

[0101] Establish a coordinate system: Use the mark information captured by the 2D camera to establish a reliable coordinate system, providing accurate positioning basis for subsequent processing, testing and other activities.

[0102] 3D camera scanning path setting: Select line laser guidance in the sub-template to enter the line laser guidance editing interface; all parameter settings in the line laser guidance setting are saved with the sub-template, and the sub-template can be placed in the main template for offset and matrix placement. You can choose to guide the dispensing inspection of the main template program one by one or perform guidance → dispensing → inspection for all main templates; in the online laser guidance editing interface, set the length, speed, scanning angle and scanning direction of the scanning path; select the guidance number and the inspection number, which represent the pre-processing storage location of the grayscale image converted from the three-dimensional height data scanned by the 3D camera. After the path is generated, the template Mark is used as the benchmark, and the actual Mark of the product will be used to compensate for the deviation of the scanning path during actual production.

[0103] It should be noted that:

[0104] 3D dispensing image preprocessing: grayscale image is processed to display product outline; select the detection area on the product; through pixel arrangement, pixel fitting and format conversion similar processing methods, when the mouse selects the selected area, a cross-sectional height enlarged image based on the shape of the selected detection area will appear on the right;

[0105] By detecting the spatial height with a 3D camera, you can select the maximum value, minimum value, first peak, and first trough related information in the capture interface as the reference point, and output the reference point or the deviation value based on the reference point, plane falling edge and other parameters.

[0106] Export dispensing template path: After preprocessing the grayscale image, the 3D camera converts it into coordinates in the 2D coordinate system. According to the dispensing coordinates, select the instruction type of the instruction and add it to the program. Set the coordinates added in the program and the corresponding instruction type as the benchmark, which will serve as the dispensing coordinate benchmark for deviation protection calculation.

[0107] It should be noted that: 3D camera coordinate conversion to 2D coordinate system grayscale image preprocessing: First, the image captured by the 3D camera is grayed to remove color information and only retain brightness information. This is because in most cases, brightness information is sufficient to meet the needs of subsequent image analysis.

[0108] Coordinate transformation: Next, each point in the processed grayscale image is transformed from the 3D space coordinates to the coordinates on the 2D plane. This process involves the concepts of projection and plane mapping, that is, projecting the points in the three-dimensional space onto the two-dimensional plane.

[0109] Serial number and coordinate association Serial number definition: Taking the serial number of 1*0 as an example, it means that this is the first point of the first storage location. The serial number here is used to uniquely identify each specific coordinate point.

[0110] Coordinate selection and addition: Users can select specific coordinates for dispensing through the command preview function. When selecting, the command type, line parameters, and the status of the switch valve must be specified. This information will be recorded and added to the program list.

[0111] Repeated addition: The same coordinate point can be added to the program list multiple times, which means that in actual operation, the point may be dispensed or otherwise processed multiple times.

[0112] Benchmark point setting and deviation protection Setting the benchmark: After completing the selection and addition of coordinates, the user can click the "Set as Benchmark" button to set the currently selected coordinate point as the benchmark point for deviation protection calculation. This is done so that in the subsequent production process, this point can be used as a reference for deviation correction.

[0113] Deviation protection: In the production process, the coordinates generated by the line laser scanning used by each device (each PCS) will be substituted into the corresponding serial number. If the actual coordinates are found to be deviated from the reference point, the dispensing position will be automatically adjusted to ensure the processing accuracy. Through the above steps, not only can the conversion from the image captured by the 3D camera to the 2D coordinates be realized, but also the coordinate points that need to be processed can be conveniently selected and managed. In addition, by setting the reference point and implementing deviation protection measures, the degree of automation in the production process and the quality stability of the finished product are greatly improved.

[0114] Post-dispensing inspection pre-processing: Detect the dispensing effect by comparing the 3D grayscale images before and after dispensing. It should be noted that the commonly used process detection abnormalities before: glue break, abnormal glue height, abnormal outer overlap, abnormal gap, abnormal glue width, and abnormal inner overlap.

[0115] The calculation method for determining the left and right edges of the glue is determined by the following parameters:

[0116] If the slope step is set to n, the actual distance represented by each pixel is d. That is, the height of the current position is h1. The height of the nth pixel to the left of the current position is h2. Then the slope of the current position k = (h1-h2) / (n*d)

[0117] Rising slope: Slope lower limit. Usually set to a negative value. The smaller the value, the faster the slope rises. When the slope k is less than this value, the position is the glue edge.

[0118] Drop height: the ratio of glue height drop. When the glue height drops beyond the set value, the calculation of the drop slope and smooth slope begins. Generally, it is set to a positive value. The larger the value, the faster the slope drops. Drop slope: the upper limit of the slope is faster. When the slope k is greater than this value, the position is the glue edge.

[0119] Smooth slope: Slope lower limit. Generally set to a positive value. The closer the value is to 0, the more gradual the slope decreases, and the closer it is to the glue dispensing plane. When the slope k is less than this value, the position is the glue edge. (The smoother, the larger the value)

[0120] Slope step: The number of pixels used when calculating the slope

[0121] The method of judging the glue break, gap, glue height, inner overlap and outer overlap is consistent with the guided point-taking method. The descending edge of the highest point of the product is used as the reference point, and the offset in the cross-sectional direction is used to confirm the point position.

[0122] The test results will be displayed on the left and right sides of the actual glue dispensing line.

[0123] Glue break: If any of the following conditions are met, the glue is broken (red line segment)

[0124] ① Glue height < glue cut height setting value and glue thickness < glue thickness setting value

[0125] ② Glue volume < glue breaking volume setting value Glue height abnormality: If any of the following conditions is met, the glue height is abnormal (sky blue line segment)

[0126] ① Glue height < less glue height setting value or glue height > more glue height setting value ② Glue volume < less glue volume setting value Outer cladding abnormality: Outer cladding height > Outer cladding height setting value (dark blue line segment)

[0127] Gap abnormality: Gap < Gap lower limit setting value or gap height > Gap upper limit setting value (brown line segment)

[0128] Abnormal glue width: glue width < glue width lower limit setting value or glue width > glue width upper limit setting value (white line segment)

[0129] Inner lining abnormality: Inner lining height > Inner lining height setting value (yellow line segment)" is explained in detail:

[0130] Principle of 3D detection after dispensing

[0131] Slope calculation method

[0132] Slope step size (n): refers to the number of pixels used when calculating the slope.

[0133] Current position height (h1): the height of the current pixel.

[0134] Height of the nth pixel (h2): The height of the nth pixel from the left.

[0135] Slope (k): By formula Calculated, where d is the actual distance represented by each pixel. Edge determination:

[0136] Rising slope: The lower limit of the slope, usually a negative value, indicates the speed at which the slope rises. When the slope k is less than this value, it is considered to have reached the glue edge.

[0137] Drop height: glue height drop ratio. When the glue height drops beyond the set value, the drop slope and smooth slope are calculated.

[0138] Downward slope: The upper limit of the slope, indicating the speed at which the slope decreases. When the slope k is greater than this value, it is considered to have reached the glue edge.

[0139] Smooth slope: The lower limit of the slope, indicating the degree of smoothness of the slope decline. When the slope k is less than this value, it is considered to have reached the glue edge.

[0140] Anomaly detection rules

[0141] Glue Cutting

[0142] Condition 1: The glue height is lower than the glue cut height setting value and the glue thickness is lower than the glue thickness setting value.

[0143] Condition 2: The glue volume is lower than the glue cut-off volume setting value.

[0144] Abnormal glue height

[0145] Condition 1: The glue height is lower than the setting value of the low glue height or higher than the setting value of the high glue height. Condition 2: The glue volume is lower than the setting value of the low glue volume.

[0146] Abnormal outer wear

[0147] The exterior height exceeds the exterior height setting value.

[0148] Abnormal clearance

[0149] The gap is less than the gap lower limit setting value or the gap height exceeds the gap upper limit setting value.

[0150] Abnormal glue width

[0151] The glue width is smaller than the lower limit setting value or larger than the upper limit setting value.

[0152] Abnormal inner layer height: The inner layer height exceeds the set value of the inner layer height.

[0153] in conclusion

[0154] By accurately calculating the glue edge and strictly monitoring various indicators, various abnormal situations in the dispensing process can be effectively identified and located to ensure product quality and production efficiency.

[0155] The present invention adds multi-angle 3D visual image processing, combined with the precise positioning of product Marks by 2D cameras, to achieve dynamic compensation of the dispensing paths of various complex structure products by multi-angle 3D cameras, to solve the problems of poor consistency of customer product assembly process, single 3D camera scanning path and uneven dispensing plane, to make the actual dispensing path completely accurate and controllable, to greatly reduce the dependence of product process assembly tolerances, to save multi-point height detection CT, to be compatible with more complex products and to freely select 2D or 3D dispensing coordinates.

[0156] It should be noted that the above-mentioned technical features are further combined with each other to form various embodiments not listed above, which are all deemed to be within the scope of the present invention; and, for ordinary technicians in this field, they can be improved or transformed according to the above description, and all these improvements and transformations should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for multi-angle rotation 3D guided dispensing detection based on two-dimensional vision, characterized in that ； Here are the steps: 3D camera rotation angle setting: calibrate the angle deviation caused by mechanical assembly Select 3D calibration path and angle: select different angles and scanning directions to calibrate 2D and 3D cameras; 3D calibration image preprocessing: According to the set angle and direction, the obtained 3D image is converted into a grayscale image for preprocessing; 2D camera VS 3D camera calibration: calibrate the coordinates of the 3D image preprocessing with the 2D camera coordinates; 2D Mark setting: record the mark points for overall offset and angle compensation of the 3D scanning path; 3D camera scanning path setting: set the 2D Mark reference coordinates to determine the 3D scanning path and angle of the template; 3D dispensing image preprocessing: preprocess the grayscale image obtained by the 3D camera; Export dispensing template path: select the path that needs to be dispensed as the template program reference point; dispensing parameter setting: set the dispensing speed, glue output, offset and dispensing height corresponding to the parameter type; Post-dispensing detection preprocessing: Detect the dispensing effect by comparing the 3D grayscale images before and after dispensing.

2. According to claim 1, the multi-angle rotation 3D guided dispensing and detection method based on two-dimensional vision is characterized in that; Before implementing the rotation angle setting of the 3D camera, it is also necessary to enable 3D image inversion and set 3D camera guide deviation protection. Enabling 3D image inversion makes it easier for customers to view 3D images; setting 3D camera guide deviation protection: according to the benchmark Mark points and benchmark mechanical dispensing coordinates of the set template, the theoretical mechanical coordinates can be inferred through the Mark coordinates and angles of different products; based on the difference between the actual glue path captured by the 3D camera of different product Marks and the theoretical mechanical coordinates, protection is performed. If the protection value is exceeded, a fitting or shutdown alarm will be issued to prevent 3D camera capture errors.

3. The multi-angle rotation 3D guided dispensing and detection method based on two-dimensional vision according to claim 1, characterized in that; The rotation angle setting of the 3D camera includes preset attitude angle and fine-tuning attitude angle; the preset attitude angles include 0°, 45° and 90°, and the rotation angle of the 3D camera can be selected according to the needs; Fine-tune the posture angle and make fine adjustments to the preset posture angle.

4. The multi-angle rotation 3D guided dispensing and detection method based on two-dimensional vision according to claim 1, characterized in that; Select the 3D calibration path and angle: select the rotation angle and scanning direction of the 3D camera; by selecting the scanning start and end points of the 3D camera, make the scanning range of the 3D camera include the entire calibration block.

5. The multi-angle rotation 3D guided dispensing and detection method based on two-dimensional vision according to claim 1, characterized in that; 3D calibration image preprocessing: Use a 3D camera device to obtain the three-dimensional contour information of the calibration block, convert the calibration block height data map into a grayscale image, and preprocess the grayscale image; perform preprocessing on the grayscale image, capture the falling edge of the side wall of the 9-point circular hole on the calibration block, and fit the center coordinates of the 9-point circular hole; after the preprocessing is completed, rescan the calibration block to obtain the absolute coordinates of the 9-point circular hole relative to the scanning path.

6. The multi-angle rotation 3D guided dispensing and detection method based on two-dimensional vision according to claim 1, characterized in that; 2D camera VS 3D camera calibration: Use a 2D camera to teach the 2D coordinates of the center of each circular hole, use circle matching to find the center positions of the 9 circular holes, and correspond them one by one with the coordinates of the 3D camera in sequence; use a 9-point calibration algorithm, establish a linear equation group based on the above 2D and 3D coordinates, and solve the calibration relationship of the 2D camera VS 3D camera.

7. The multi-angle rotation 3D guided dispensing and detection method based on two-dimensional vision according to claim 1 is characterized in that ; 2D Mark setting: Use a 2D camera to locate the mark and establish a coordinate system.

8. The multi-angle rotation 3D guided dispensing and detection method based on two-dimensional vision according to claim 1, characterized in that; 3D camera scanning path setting: Select line laser guidance in the sub-template to enter the line laser guidance editing interface; all parameter settings in the line laser guidance setting are saved with the sub-template, and the sub-template can be placed in the main template for offset and matrix placement. You can choose to guide the dispensing inspection of the main template program one by one or perform guidance → dispensing → inspection for all main templates; in the online laser guidance editing interface, set the length, speed, scanning angle and scanning direction of the scanning path; select the guidance number and the inspection number, which represent the pre-processing storage location of the grayscale image converted from the three-dimensional height data scanned by the 3D camera. After the path is generated, the template Mark is used as the benchmark, and the actual Mark of the product will be used to compensate for the deviation of the scanning path during actual production.

9. The multi-angle rotation 3D guided dispensing and detection method based on two-dimensional vision according to claim 1, characterized in that; 3D dispensing image preprocessing: grayscale image is processed to display product outline; select the detection area on the product; through pixel arrangement, pixel fitting and format conversion similar processing methods, when the mouse selects the selected area, a cross-sectional height enlarged image based on the shape of the selected detection area will appear on the right; By detecting the spatial height with a 3D camera, you can select the maximum value, minimum value, first peak, and first trough related information in the capture interface as the reference point, and output the reference point or the deviation value based on the reference point, plane falling edge and other parameters.

10. The multi-angle rotation 3D guided dispensing and detection method based on two-dimensional vision according to claim 1, characterized in that; Export dispensing template path: After preprocessing the grayscale image, the 3D camera converts it into coordinates in the 2D coordinate system. According to the dispensing coordinates, select the instruction type of the instruction and add it to the program. Set the coordinates added in the program and the corresponding instruction type as the benchmark, which will serve as the dispensing coordinate benchmark for deviation protection calculation.

Images