Method and system for high-precision automatic glue spraying based on machine vision

Abstract

The present invention provides a method and system for high-precision automatic glue spraying based on machine vision, which uses a three-dimensional camera to perform point cloud acquisition, preprocessing and target recognition to extract a point cloud model of each car interior; the point cloud model of the car interior is compared with multiple stored historical point cloud models to match the historical point cloud model with the highest similarity, and further obtain the corresponding historical glue spraying trajectory and parameters. Based on the historical glue spraying trajectory parameters, a glue spraying robot performs automatic glue spraying on multiple car interiors in sequence. Since the method of the present invention automatically obtains the glue spraying trajectory and parameters based on real-time three-dimensional images and automatic model matching, there is no need to manually switch the glue spraying trajectory and parameters according to the type of car interior, and there is no need to manually teach the glue spraying robot, thereby improving production efficiency. In addition, the glue spraying trajectory and parameters can be iteratively optimized through multiple glue spraying results to continuously improve the glue spraying accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent glue spraying, and in particular relates to a method and system for high-precision automatic glue spraying based on machine vision. Background Art

[0002] Car interiors are car products used inside cars, including car seat cushions, car dashboards, car trunk pads, etc. The production process of car interiors is generally to compound leather or fabrics of specific materials on the formed car interior workpieces to form the final car interior products. In the above-mentioned compounding process, it is necessary to spray glue on the surface of the car interior workpiece. There are many kinds of car interior products, and accordingly, there are many different shapes of car interiors that need to be sprayed. In actual production, the production of car interiors is usually a multi-variety, small-batch production. Therefore, the current automatic spraying of car interiors has the following problems: First, after a small batch of car interior spraying is completed, the product needs to be switched. Accordingly, the operator needs to reset the spraying program and re-teach the robot's spraying trajectory, which is cumbersome; second, the above switching process requires the operator to manually teach near the robot and adjust the spraying parameters back and forth between the robot and the operating terminal, so it is very time-consuming, and each switch takes one to two hours; third, multiple switches are often required during a day's production process, so the time spent on the switching process accounts for almost half of the total production time, and the production efficiency is low. At the same time, with the development of technology and product upgrades, the precision requirements for glue spraying are becoming higher and higher. Therefore, the existing automated glue spraying technology is difficult to meet the high-precision glue spraying needs of the above-mentioned various automotive interiors.

[0003] The guidance of 3D machine vision enables more precise control of the robot, which is expected to achieve high-precision automated glue spraying. However, there is currently a lack of methods for automated glue spraying of automotive interiors based on 3D machine vision. Summary of the invention

[0004] In order to solve the above problems, a method and system for high-precision automatic glue spraying based on machine vision are provided. The present invention adopts the following technical solutions:

[0005] The present invention provides a method for high-precision automatic glue spraying based on machine vision, which performs high-precision glue spraying operations on multiple workpieces to be glued placed in a predetermined working area through a glue spraying robot based on machine vision, and is characterized in that it includes step S1, point cloud acquisition of the predetermined working area through a three-dimensional camera to obtain point cloud data; step S2, preprocessing the point cloud data to obtain a scene point cloud; step S3, point cloud clustering and segmenting the scene point cloud to obtain a point cloud model corresponding to each car interior; step S4, for each car interior, the point cloud model is respectively compared with multiple historical point cloud models to obtain the historical point cloud model with the highest similarity; step S5, for each car interior, the historical glue spraying trajectory and the historical glue spraying trajectory parameters corresponding to the historical point cloud model are obtained; step S6, glue spraying operations are performed on multiple car interiors in sequence according to a preset order, specifically, for each car interior, glue spraying operations are performed by the glue spraying robot based on the historical glue spraying trajectory and the historical glue spraying trajectory parameters.

[0006] The method for high-precision automatic glue spraying based on machine vision provided by the present invention may also have such a technical feature, wherein step S4 includes the following sub-steps: step S4-1, resampling the point cloud model and the historical point cloud model to obtain the current model key points and the historical model key points respectively; step S4-2, calculating the current model descriptor based on the current model key points, and calculating the historical model descriptor based on the historical model key points; step S4-3, for each historical point cloud model, based on the current model descriptor and the historical descriptor, point cloud matching is performed between the point cloud model and the historical point cloud model to obtain matching point pairs and similarities; step S4-4, obtaining the historical point cloud model with the highest similarity and the corresponding matching point pairs; step S4-5, calculating the transformation matrix between the point cloud model and the historical point cloud model based on the matching point pairs.

[0007] The method for high-precision automatic glue spraying based on machine vision provided by the present invention may also have such technical features, wherein step S6 includes the following sub-steps: step S6-1, for each automobile interior, converting the historical glue spraying trajectory into the current glue spraying trajectory based on the conversion matrix, and converting the historical glue spraying trajectory parameters into the current glue spraying trajectory parameters based on the conversion matrix; step S6-2, connecting multiple current glue spraying trajectories in sequence according to a preset order to obtain a final glue spraying trajectory, and merging multiple current glue spraying trajectory parameters in sequence according to a preset order to obtain the final glue spraying trajectory parameters; step S6-3, based on the final glue spraying trajectory and the final glue spraying trajectory parameters, performing glue spraying operations through a glue spraying robot.

[0008] The method for high-precision automatic glue spraying based on machine vision provided by the present invention may also have such a technical feature, wherein the historical glue spraying trajectory and the historical glue spraying trajectory parameters are optimized and updated according to the glue spraying results, specifically including the following steps: Step A1, making corresponding corrections to the current glue spraying trajectory and the current glue spraying trajectory parameters according to the glue spraying results; Step A2, converting the current glue spraying trajectory and the current glue spraying trajectory parameters to the coordinate system of the historical glue spraying trajectory through a conversion matrix; Step A3, updating the historical glue spraying trajectory to the current glue spraying trajectory, and updating the historical glue spraying trajectory parameters to the current glue spraying trajectory parameters.

[0009] The method for high-precision automated glue spraying based on machine vision provided by the present invention may also have such a technical feature, wherein step S1 includes the following sub-steps: step S1-1, photographing the workpiece multiple times from different viewpoints with a three-dimensional camera to obtain multiple depth maps; step S1-2, converting the multiple depth maps into multiple partial point clouds respectively; wherein the multiple depth maps completely cover the predetermined working area, and the point cloud data is a collection of multiple partial point clouds.

[0010] The method for high-precision automatic glue spraying based on machine vision provided by the present invention may also have such technical features, wherein step S2 includes the following sub-steps: step S2-1, performing point cloud filtering on the point cloud data; step S2-2, down-sampling the point cloud data; step S2-3, performing feature description and extraction on the point cloud data to obtain point cloud features; step S2-4, based on the point cloud features, performing point cloud splicing on the point cloud data to obtain a scene point cloud; step S2-5, based on the point cloud features, performing point cloud segmentation on the scene point cloud to remove point clouds other than multiple automobile interiors.

[0011] The method for high-precision automatic glue spraying based on machine vision provided by the present invention may also have such a technical feature, wherein the point cloud clustering segmentation adopts the Euclidean point cloud clustering segmentation method.

[0012] The method for high-precision automatic glue spraying based on machine vision provided by the present invention may also have such a technical feature, wherein the historical glue spraying trajectory parameters include the trajectory direction and the trajectory line spacing.

[0013] The method for high-precision automated glue spraying based on machine vision provided by the present invention may also have such a technical feature, wherein the three-dimensional camera adopts any one or a combination of several of the binocular vision solution, three-dimensional structured light solution, TOF solution, and laser triangulation solution.

[0014] The present invention provides a system for high-precision automatic glue spraying based on machine vision, which is characterized by comprising a 3D camera for 3D photographing of the automobile interior to be sprayed with glue; a glue spraying robot for spraying glue on the automobile interior; and a control device for controlling the process of 3D photographing and glue spraying, wherein the control device comprises: a point cloud acquisition unit for acquiring a scene point cloud containing the automobile interior through the 3D camera and preprocessing the scene point cloud; a history model storage unit for storing a plurality of history point cloud models; a target recognition unit for recognizing each automobile interior from the scene point cloud and obtaining the location of each automobile interior. a corresponding point cloud model; a model comparison unit, which compares the point cloud model with each historical point cloud model respectively to obtain the historical point cloud model with the highest similarity; a glue spraying trajectory generation unit, which obtains the historical glue spraying trajectory and the historical glue spraying trajectory parameters corresponding to the historical point cloud model with the highest similarity, and converts the historical glue spraying trajectory into a glue spraying trajectory based on the current glue spraying robot coordinate system; a glue spraying control unit, which controls the glue spraying robot to spray glue on the car interior in sequence according to the glue spraying trajectory and the historical glue spraying trajectory parameters; and a glue spraying trajectory optimization unit, which optimizes and updates the historical glue spraying trajectory and the historical glue spraying trajectory parameters according to the glue spraying results.

[0015] Function and Effect of the Invention

[0016] According to the method for high-precision automatic glue spraying based on machine vision of the present invention, a point cloud is collected and preprocessed for a predetermined working area by a three-dimensional camera, and a scene point cloud containing three-dimensional information of all car interiors in the area is obtained; the point cloud model of each car interior is extracted by performing point cloud clustering and segmentation on the scene point cloud; the point cloud model of the car interior is compared with a plurality of stored historical point cloud models, and the historical point cloud model with the highest similarity is matched, and the corresponding historical glue spraying trajectory and historical glue spraying trajectory parameters are further obtained, and based on the historical glue spraying trajectory and historical glue spraying trajectory parameters, the glue spraying robot can perform automatic glue spraying on a plurality of car interiors in sequence. Since the method of the present invention automatically obtains the historical glue spraying trajectory and parameters corresponding to each car interior based on real-time three-dimensional images and automatic model matching, there is no need to manually switch the glue spraying trajectory and parameters according to the type of car interior, and there is no need to manually teach the glue spraying robot, which saves a lot of time and greatly improves the production efficiency of car interiors. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a flow chart of a method for high-precision automated glue spraying based on machine vision in an embodiment of the present invention;

[0018] Figure 2 is a flow chart of step S1 of a method for high-precision automated glue spraying based on machine vision in an embodiment of the present invention;

[0019] Figure 3 is a flow chart of step S2 of the method for high-precision automatic glue spraying based on machine vision in an embodiment of the present invention;

[0020] Figure 4 is a flow chart of step S4 of the method for high-precision automated glue spraying based on machine vision in an embodiment of the present invention;

[0021] Figure 5 is a flow chart of step S6 of the method for high-precision automated glue spraying based on machine vision in an embodiment of the present invention;

[0022] Figure 6 is a flow chart of optimizing and updating the historical glue spraying trajectory and the parameters of the historical glue spraying trajectory in an embodiment of the present invention;

[0023] Figure 7 It is a structural block diagram of a system for high-precision automatic glue spraying based on machine vision in an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the method and system for high-precision automatic glue spraying based on machine vision of the present invention are specifically described below in combination with embodiments and drawings.

[0025] <Example>

[0026] In this embodiment, there are two car interiors to be sprayed with glue, namely, a car seat cushion and a car dashboard. The upper surface of the car seat cushion is roughly a saddle surface, and the upper surface of the car dashboard is roughly a single curved surface with multiple recesses, holes and other structures. The range to be sprayed with glue is the entire upper surface of the two car interiors. The two car interiors are fixed and placed in the predetermined working area by a bracket at the same time.

[0027] In this embodiment, the three-dimensional camera is a three-dimensional structured light camera. The structured light source in the three-dimensional camera projects black and white structured light. After the structured light is projected onto the surface of an object, the black and white stripes or spots generated by the structured light produce different degrees of deformation according to the shape of the object surface. The three-dimensional camera can calculate the position information of the surface points of the object based on the deformation, thereby obtaining a depth map of the object.

[0028] In this embodiment, the glue spraying robot has a six-axis mechanical arm, and thus has six-axis degrees of freedom, and can realize glue spraying operations of multiple trajectories and multiple different angles. At the same time, an automatic spray gun is provided at the end of the glue spraying robot, and the automatic spray gun is fluidically connected to a glue storage device storing glue liquid for glue spraying, and is used to spray the glue liquid. The three-dimensional camera is fixedly mounted on the six-axis flange of the mechanical arm of the glue spraying robot.

[0029] In this embodiment, the control device is an industrial computer, which is installed with 3D visual spraying software, PCL (Point Cloud Library) library and OMPL (Open Motion Planning Library) library. The 3D visual spraying software includes UI interface, robot module, hand-eye calibration module, 3D vision module, 3D point cloud processing module and spraying process package. The industrial computer is connected to the 3D camera and the glue spraying robot respectively, and is used to control the 3D camera to collect point clouds of the car interior to be sprayed with glue, and control the glue spraying robot to spray glue on the car interior.

[0030] The industrial computer stores multiple historical point cloud models, including point cloud models of car interiors such as car seat cushions, car dashboards, and car door armrests. The industrial computer also stores multiple sets of historical glue spraying trajectories and historical glue spraying trajectory parameters, which correspond to multiple historical point cloud models.

[0031] This embodiment first uses a 3D camera to collect point clouds of multiple car interiors to be sprayed; then, the collected scene point clouds are preprocessed; then, the point cloud models of each car interior are extracted from the scene point clouds; then, the historical point cloud model with the highest similarity is found through comparison; further, the historical spraying trajectory corresponding to the historical point cloud model is obtained; finally, the glue is sprayed according to the historical glue spraying trajectory; in addition, the historical glue spraying trajectory can also be optimized and updated according to the glue spraying results. The overall process includes 7 processes: collecting 3D point clouds, preprocessing, target recognition, model comparison, obtaining glue spraying trajectory, performing glue spraying, and optimizing glue spraying trajectory.

[0032] Figure 1 It is a flow chart of a method for high-precision automated glue spraying based on machine vision in an embodiment of the invention.

[0033] like Figure 1 As shown, the method for high-precision automatic glue spraying based on machine vision includes the following steps:

[0034] Step S1, collecting point cloud data of a predetermined working area through a three-dimensional camera.

[0035] Figure 2 4 is a flow chart of step S1 of a method for high-precision automated glue spraying based on machine vision in an embodiment of the present invention.

[0036] like Figure 2 As shown, step S1 of this embodiment specifically includes the following sub-steps:

[0037] Step S1-1, taking multiple shots from different viewpoints using a 3D camera to obtain multiple depth maps;

[0038] Step S1-2, converting multiple depth maps into multiple partial point clouds.

[0039] Among them, multiple depth maps completely cover the predetermined working area, and the point cloud data is a collection of multiple partial point clouds.

[0040] In this embodiment, the predetermined working area is photographed from four preset viewpoints by a three-dimensional camera to obtain four depth maps, which completely cover the entire predetermined working area, that is, completely cover the two car interiors to be sprayed with glue.

[0041] Then, based on the camera internal parameters, the four depth maps are converted into four partial point clouds through the existing conversion method. The collection of these four partial point clouds is point cloud data containing two three-dimensional information of the car interior. The three-dimensional point cloud data contains information such as spatial resolution, point position accuracy, surface normal vector, etc., which can express the spatial outline and specific position of the object.

[0042] Step S2: pre-process the point cloud data to obtain a scene point cloud.

[0043] Figure 3 4 is a flow chart of step S2 of the method for high-precision automatic glue spraying based on machine vision in an embodiment of the present invention.

[0044] like Figure 3 As shown, step S2 of this embodiment specifically includes the following sub-steps:

[0045] Step S2-1, performing point cloud filtering on the point cloud data;

[0046] Step S2-2, downsampling the point cloud data;

[0047] Step S2-3, performing feature description and extraction on the point cloud data to obtain point cloud features;

[0048] Step S2-4, based on the point cloud features, point cloud data is stitched to obtain a scene point cloud;

[0049] Step S2-5, based on the point cloud features, the scene point cloud is segmented to remove the point cloud other than the outline of the workpiece.

[0050] In this embodiment, the point cloud filtering, downsampling and point cloud feature extraction methods are all existing algorithms in the PCL library.

[0051] Point cloud filtering can be used to smooth the 3D point cloud, solve the problem of irregular density of 3D point cloud data, and remove noise data such as outliers.

[0052] Downsampling can reduce the number of points in the three-dimensional point cloud, that is, reduce the point cloud data, while maintaining the shape characteristics of the point cloud, which can reduce the amount of calculation for subsequent calculations and improve the calculation accuracy to a certain extent.

[0053] Point cloud features of a three-dimensional point cloud can be obtained through feature description and extraction. In this embodiment, the point cloud features include point cloud single point features and point cloud local features.

[0054] In this embodiment, the point cloud stitching adopts the ICP closest point iteration algorithm in the PCL library to convert multiple partial point clouds into the same coordinate system, overlap some of the same points in different partial point clouds based on the point cloud features, and thus stitch the four partial point clouds into a complete scene point cloud.

[0055] The point cloud segmentation adopts the RANSAC random sampling consensus algorithm in the PCL library, which can detect objects with special shapes from cluttered point clouds. In the scene point cloud of this embodiment, the background except for the two car interiors is basically flat. Therefore, this algorithm can well remove the background part in the scene point cloud while retaining the two car interior parts.

[0056] Therefore, through step S2 of this embodiment, a scene point cloud containing three-dimensional information of two car interiors is obtained.

[0057] Step S3, performing point cloud clustering and segmentation on the scene point cloud to obtain a point cloud model of each car interior.

[0058] In this embodiment, the point cloud clustering segmentation adopts the Euclidean clustering segmentation algorithm in the PCL library, takes the Euclidean distance as the distance judgment standard, finds the nearest point through the KD-Tree nearest neighbor search algorithm, and clusters the points less than the set threshold among the nearest points into a set, thereby clustering to obtain point cloud models of two car interiors.

[0059] Step S4: for each car interior, the point cloud model is compared with a plurality of historical point cloud models to obtain the historical point cloud model with the highest similarity.

[0060] Figure 4 4 is a flow chart of step S4 of the method for high-precision automatic glue spraying based on machine vision in an embodiment of the present invention.

[0061] like Figure 4 As shown, step S4 of this embodiment specifically includes the following sub-steps:

[0062] Step S4-1, resampling the point cloud model and the historical point cloud model respectively to obtain the key points of the current model and the key points of the historical model;

[0063] Step S4-2, calculating the current model descriptor based on the current model key points, and calculating the historical model descriptor based on the historical model key points;

[0064] Step S4-3, for each historical point cloud model, based on the current model descriptor and the historical model descriptor, the point cloud model is matched with the historical point cloud model to obtain matching point pairs and similarities;

[0065] Step S4-4, obtaining the historical point cloud model with the highest similarity and the corresponding matching point pair;

[0066] Step S4-5, based on the matching point pairs, calculate the transformation matrix between the point cloud model and the historical point cloud model.

[0067] In this embodiment, the resampling is a uniform sampling method in the PCL library. Uniform sampling can effectively reduce noise data in the point cloud without changing the position of the point in the point cloud, and is therefore suitable for extracting key points.

[0068] The model descriptor is a SHOT (Histogram of Normal Orientation) descriptor, which can be calculated using an existing algorithm in the PCL library.

[0069] Point cloud matching uses the KD-Tree nearest neighbor search algorithm in the PCL library. In the point cloud model, a current model key point with the most similar model descriptor is matched to each historical model key point of the historical point cloud model, thereby obtaining multiple point pairs. Point pairs with model descriptors that are too far apart are removed from the multiple point pairs to obtain multiple matching point pairs. The number of matching point pairs can be used as the similarity between the two models.

[0070] The transformation matrix is ​​calculated by the existing point cloud conversion method in the PCL library. The transformation matrix includes the rotation matrix and the translation matrix. The transformation matrix can be used to transform the point cloud model to the historical point cloud model.

[0071] In this embodiment, taking a car seat cushion as an example, the point cloud model of the car seat cushion is matched with multiple stored historical point cloud models respectively. The historical point cloud models include point cloud models of a car seat cushion, a car dashboard, a car door armrest, etc. Through the above calculation and point cloud matching, the historical point cloud model of the car seat cushion has the largest number of matching point pairs. Therefore, the historical point cloud model of the car seat cushion is used as the historical point cloud model with the highest similarity, and the above transformation matrix is ​​calculated.

[0072] As described above, through step S4 of this embodiment, the historical point cloud model with the highest similarity is matched, and the transformation matrix between the two models is obtained.

[0073] Step S5, for each car interior, obtaining the historical glue spraying trajectory and the historical glue spraying trajectory parameters corresponding to the historical point cloud model with the highest similarity.

[0074] In this embodiment, the historical point cloud model, the historical glue spraying trajectory and the historical glue spraying trajectory parameters in the industrial computer are stored correspondingly, so after finding the historical point cloud model with the highest similarity, the historical glue spraying trajectory and parameters can be directly read.

[0075] Step S6, performing the glue spraying operation on the plurality of automobile interiors in sequence according to a preset order. Specifically, for each automobile interior, the glue spraying operation is performed by the glue spraying robot based on the historical glue spraying trajectory and the historical glue spraying trajectory parameters.

[0076] Figure 5 4 is a flow chart of step S5 of the method for high-precision automatic glue spraying based on machine vision in an embodiment of the present invention.

[0077] like Figure 5 As shown, step S6 of this embodiment specifically includes the following sub-steps:

[0078] Step S6-1, for each car interior, convert the historical glue spraying trajectory into the current glue spraying trajectory through the transformation matrix, and convert the historical glue spraying trajectory parameters into the current glue spraying trajectory parameters through the transformation matrix;

[0079] Step S6-2, sequentially connecting the multiple current glue spraying trajectories in a preset order to obtain a final glue spraying trajectory, and sequentially merging the multiple current glue spraying trajectory parameters in a preset order to obtain a final glue spraying trajectory parameter;

[0080] Step S6-3, based on the final glue spraying trajectory and the final glue spraying trajectory parameters, the glue spraying operation is performed by the glue spraying robot.

[0081] In this embodiment, due to factors such as equipment conditions and environmental influences, there will inevitably be some differences in distance and angle between the point cloud collected this time and the point cloud collected historically. Therefore, it is necessary to use the above-mentioned transformation matrix to convert the historical glue spraying trajectory into the robot coordinate system corresponding to the current point cloud model.

[0082] The historical glue spraying trajectory parameters include trajectory direction and trajectory line spacing, etc., so they also need to be converted through the above-mentioned transformation matrix.

[0083] In this embodiment, the position of the glue spraying robot is used as a reference, and the pre-set order is from near to far.

[0084] As described above, through real-time 3D shooting, automatic model matching and automatic glue spraying trajectory acquisition, two different car interiors were automatically sprayed with glue in turn.

[0085] In addition, in this embodiment, the operator can also optimize and update the historical glue spraying trajectory and the historical glue spraying trajectory parameters according to the glue spraying results. The optimization and update can be performed asynchronously with the above-mentioned glue spraying process without affecting the glue spraying process.

[0086] Figure 6 It is a flow chart of historical glue spraying trajectory and parameter optimization and updating in an embodiment of the present invention.

[0087] like Figure 6 As shown, in this embodiment, the optimization and update of the historical glue spraying trajectory and the parameters of the historical glue spraying trajectory specifically include the following steps:

[0088] Step A1, optimizing the current glue spraying trajectory and parameters of the current glue spraying trajectory according to the glue spraying result;

[0089] Step A2, converting the optimized current glue spraying trajectory and the parameters of the current glue spraying trajectory into the coordinate system of the historical glue spraying trajectory through a transformation matrix;

[0090] Step A3, updating the historical glue spraying trajectory to the current glue spraying trajectory, and updating the historical glue spraying trajectory parameters to the current glue spraying trajectory parameters.

[0091] In this embodiment, the glue spraying results include the following problems: one corner of the car interior is not covered by the glue spraying, and the overall glue layer is slightly thick. According to the above problems, the operator optimizes the corresponding trajectory and parameters through the three-dimensional visual spraying software on the industrial computer. Specifically, for the problem that one corner is not covered, the operator manually edits the glue spraying trajectory through the UI interface, adds key points at the corresponding position, and generates additional glue spraying trajectory segments through the software; for the problem that the overall glue layer is slightly thick, the operator enters a smaller value in the flow column in the glue spraying parameters through the UI interface. Then, the operator converts the optimized glue spraying trajectory and parameters to the coordinate system of the historical glue spraying trajectory through the above-mentioned transformation matrix, and stores the converted glue spraying trajectory and parameters as new historical glue spraying trajectory and new historical glue spraying trajectory parameters in the industrial computer for use in the next glue spraying.

[0092] This embodiment also provides a system for high-precision automated glue spraying based on machine vision.

[0093] Figure 7 It is a structural block diagram of a system for high-precision automatic glue spraying based on machine vision in an embodiment of the present invention.

[0094] like Figure 7 As shown, the system 10 for high-precision automatic glue spraying based on machine vision according to the embodiment of the present invention includes the above-mentioned three-dimensional camera 11 , the above-mentioned glue spraying robot 12 and the above-mentioned control device 13 .

[0095] The control device 13 includes a main control unit 131 , a history model storage unit 132 , a point cloud acquisition unit 133 , a target recognition unit 134 , a model comparison unit 135 , a glue spraying trajectory generation unit 136 , a glue spraying control unit 137 and a glue spraying trajectory optimization unit 138 .

[0096] The historical model storage unit 132 is used to store the above-mentioned multiple historical point cloud models.

[0097] The point cloud acquisition unit 133 uses the method of step S1 - 2 to acquire the scene point cloud including the automobile interior and perform preprocessing.

[0098] The target recognition unit 134 uses the method of step S3 to recognize each car interior from the scene point cloud and obtain a point cloud model of each car interior.

[0099] The model comparison unit 135 uses the method of step S4 to compare the point cloud model of the automobile interior with the historical point cloud model stored in the historical model storage unit 132 to obtain the historical point cloud model with the highest similarity.

[0100] The glue spraying trajectory generating unit 136 uses the method of step S5 to obtain the historical glue spraying trajectory and historical glue spraying trajectory parameters corresponding to the historical point cloud model with the highest similarity, and converts the historical glue spraying trajectory and historical glue spraying trajectory parameters to the coordinate system of the glue spraying robot 12 .

[0101] The glue spraying control unit 137 uses the method of step S6 to control the glue spraying robot 12 to perform glue spraying operations on the automobile interior in sequence according to the historical glue spraying trajectory and the historical glue spraying trajectory parameters.

[0102] The glue spraying trajectory optimization unit 138 optimizes and updates the historical glue spraying trajectory and the historical glue spraying trajectory parameters using the above-mentioned optimization and update methods.

[0103] Example Function and Effect

[0104] According to the method and system for high-precision automatic glue spraying based on machine vision provided by this embodiment, a point cloud is collected and preprocessed for a predetermined working area by a three-dimensional camera, and a scene point cloud containing three-dimensional information of all car interiors in the area is obtained; the point cloud model of each car interior is extracted by performing point cloud clustering and segmentation on the scene point cloud; the point cloud model of the car interior is compared with multiple stored historical point cloud models, and the historical point cloud model with the highest similarity is matched, and the corresponding historical glue spraying trajectory and historical glue spraying trajectory parameters are further obtained. Based on the historical glue spraying trajectory and historical glue spraying trajectory parameters, the glue spraying robot can perform automatic glue spraying on multiple car interiors in sequence. Since the method of the present invention automatically obtains the glue spraying trajectory and parameters corresponding to each car interior based on real-time three-dimensional images and automatic model matching, there is no need to manually switch the glue spraying trajectory and parameters according to the type of car interior, and there is no need to manually teach the glue spraying robot, which saves a lot of time and greatly improves the production efficiency of car interiors.

[0105] Specifically, the point cloud model and the historical point cloud model of the car interior are resampled to obtain the model key points of the two models respectively, and the model descriptors are obtained based on the model key points respectively. The point cloud model and the historical point cloud model are matched based on the model descriptors. For each historical point cloud model, matching point pairs and similarities are obtained. Therefore, the historical point cloud model with the highest similarity can be obtained through comparison, and the corresponding historical glue spraying trajectory can be further obtained. The conversion matrix can be calculated, so that the historical glue spraying trajectory can be converted to the current robot coordinate system through the conversion matrix and used. The above method can automatically obtain high-precision historical glue spraying trajectory, without the need to collect high-precision and high-density three-dimensional point clouds in real time, and there is no need to reconstruct the three-dimensional point cloud into a surface model to regenerate the glue spraying trajectory. Therefore, a large amount of real-time calculations are saved, which greatly improves production efficiency and reduces energy consumption of calculations.

[0106] Moreover, since the method of the present invention also optimizes and updates the historical glue spraying trajectory and parameters according to the glue spraying results, the accuracy of the next automatic glue spraying can be improved, and continuous iterative optimization can be performed through multiple glue spraying results, and the glue spraying accuracy can be continuously improved. At the same time, the optimization and update of the glue spraying trajectory and parameters can be performed asynchronously with production, without affecting the production efficiency of automotive interiors.

[0107] The above embodiments are only used to illustrate specific implementation modes of the present invention, and the present invention is not limited to the description scope of the above embodiments.

[0108] In this embodiment, the optimization of the historical glue spraying trajectory and parameters is manually performed by the operator. In practical applications, it can also be completed by an automated method. For example, a fluorescent agent can be added to the glue solution, and the fluorescent trajectory of the glue spraying can be obtained by photographing. The fluorescent trajectory can be compared with the pre-stored ideal trajectory through image processing and comparison methods, and the glue spraying trajectory and parameters can be automatically optimized according to the comparison results.

[0109] In this embodiment, the 3D camera adopts a 3D structured light solution. In practical applications, the 3D camera may also adopt a binocular vision solution, a TOF solution, a laser triangulation solution, etc., or a combination of multiple technologies, such as a combination of binocular vision and 3D structured light.

[0110] In this embodiment, the area that needs to be sprayed with glue is the entire upper surface of the car interior. In actual applications, the area that needs to be sprayed with glue may also be part of the upper surface of the car interior. The operator can manually select the glue spraying area in the point cloud model of the car interior.

Claims

1. A method for high-precision automatic glue spraying based on machine vision, wherein a glue spraying robot performs high-precision glue spraying on a plurality of automobile interiors to be glued placed in a predetermined working area, characterized in that: include: Step S1, collecting point cloud data of the predetermined working area by a three-dimensional camera; Step S2, preprocessing the point cloud data to obtain a scene point cloud; Step S3, performing point cloud clustering and segmentation on the scene point cloud to obtain a point cloud model of each of the car interiors; Step S4, for each of the automobile interiors, comparing the point cloud model with a plurality of historical point cloud models, and obtaining the historical point cloud model with the highest similarity; Step S5, for each of the automobile interiors, obtaining the historical glue spraying trajectory and the historical glue spraying trajectory parameters corresponding to the historical point cloud model with the highest similarity; Step S6, performing the glue spraying operation on the plurality of automobile interiors in sequence according to a preset order. Specifically, for each automobile interior, the glue spraying operation is performed by the glue spraying robot based on the historical glue spraying trajectory and the historical glue spraying trajectory parameters. Wherein, the step S4 includes the following sub-steps: Step S4-1, resampling the point cloud model and the historical point cloud model respectively to obtain current model key points and historical model key points; Step S4-2, calculating a current model descriptor based on the current model key points, and calculating a historical model descriptor based on the historical model key points; Step S4-3, for each of the historical point cloud models, based on the current model descriptor and the historical descriptor, performing point cloud matching between the point cloud model and the historical point cloud model to obtain matching point pairs and similarities; Step S4-4, obtaining the historical point cloud model with the highest similarity and the corresponding matching point pair; Step S4-5, based on the matching point pairs, calculating the transformation matrix between the point cloud model and the historical point cloud model.

2. The method for high-precision automatic glue spraying based on machine vision according to claim 1, Features: in, The step S6 comprises the following sub-steps: Step S6-1, for each of the automobile interior trims, converting the historical glue spraying trajectory into the current glue spraying trajectory through the transformation matrix, and converting the historical glue spraying trajectory parameters into the current glue spraying trajectory parameters through the transformation matrix; Step S6-2, sequentially connecting the multiple current glue spraying trajectories in a preset order to obtain a final glue spraying trajectory, and sequentially merging the multiple current glue spraying trajectory parameters in a preset order to obtain a final glue spraying trajectory parameter; Step S6-3: Based on the final glue spraying trajectory and the final glue spraying trajectory parameters, the glue spraying operation is performed by the glue spraying robot.

3. The method for high-precision automatic glue spraying based on machine vision according to claim 2, Features: in, Optimizing and updating the historical glue spraying trajectory and the historical glue spraying trajectory parameters according to the glue spraying result specifically includes the following steps: Step A1, optimizing the current glue spraying trajectory and the parameters of the current glue spraying trajectory according to the glue spraying result; Step A2, converting the optimized current glue spraying trajectory and the current glue spraying trajectory parameters into the coordinate system of the historical glue spraying trajectory through the transformation matrix; Step A3, updating the historical glue spraying trajectory to the current glue spraying trajectory, and updating the historical glue spraying trajectory parameters to the current glue spraying trajectory parameters.

4. The method for high-precision automatic glue spraying based on machine vision according to claim 1, characterized in that: in, The step S1 includes the following sub-steps: Step S1-1, taking multiple shots from different viewpoints using the 3D camera to obtain multiple depth maps; Step S1-2, converting the plurality of depth maps into a plurality of partial point clouds respectively; wherein the plurality of depth maps completely cover the predetermined working area, The point cloud data is a collection of multiple partial point clouds.

5. The method for high-precision automatic glue spraying based on machine vision according to claim 1, Features: in, The step S2 includes the following sub-steps: Step S2-1, performing point cloud filtering on the point cloud data; Step S2-2, downsampling the point cloud data; Step S2-3, performing feature description and extraction on the point cloud data to obtain point cloud features; Step S2-4, based on the point cloud features, performing point cloud stitching on the point cloud data to obtain the scene point cloud; Step S2-5: performing point cloud segmentation on the scene point cloud based on the point cloud features, and removing point clouds other than the multiple car interiors.

6. The method for high-precision automatic glue spraying based on machine vision according to claim 1, characterized in that: in, The point cloud clustering segmentation adopts the Euclidean point cloud clustering segmentation method.

7. The method for high-precision automatic glue spraying based on machine vision according to claim 1, characterized in that: in, The historical glue spraying trajectory parameters include trajectory direction and trajectory line spacing.

8. The method for high-precision automatic glue spraying based on machine vision according to claim 1, characterized in that: in, The three-dimensional camera adopts any one or a combination of binocular vision solution, three-dimensional structured light solution, TOF solution, and laser triangulation solution.

9. A system for high-precision automatic glue spraying based on machine vision, characterized in that: include: A 3D camera is used to take 3D images of the car interior to be sprayed with glue; A glue spraying robot, used for performing glue spraying operation on the automobile interior; as well as A control device is used to control the process of the three-dimensional camera and the glue spraying operation, Wherein, the control device comprises: A point cloud acquisition unit, which acquires a scene point cloud including the automobile interior through the three-dimensional camera and pre-processes the scene point cloud; A historical model storage unit stores a plurality of historical point cloud models; A target recognition unit, which recognizes each of the car interiors from the scene point cloud and obtains a point cloud model of each of the car interiors; A model comparison unit compares the point cloud model with each of the historical point cloud models to obtain the historical point cloud model with the highest similarity; A glue spraying trajectory generating unit, which obtains a historical glue spraying trajectory and historical glue spraying trajectory parameters corresponding to the historical point cloud model with the highest similarity, and converts the historical glue spraying trajectory into a coordinate system of the glue spraying robot; a glue spraying control unit, controlling the glue spraying robot to sequentially perform the glue spraying operation on the automobile interior according to the historical glue spraying trajectory and the historical glue spraying trajectory parameters; and The glue spraying trajectory optimization unit optimizes and updates the historical glue spraying trajectory and the historical glue spraying trajectory parameters according to the glue spraying results. Wherein, the model comparison unit performs comparison according to the following steps: Resampling the point cloud model and the historical point cloud model respectively to obtain current model key points and historical model key points; Calculate the current model descriptor based on the current model key points, and calculate the historical model descriptor based on the historical model key points; For each of the historical point cloud models, based on the current model descriptor and the historical descriptor, the point cloud model is matched with the historical point cloud model to obtain matching point pairs and similarities; Obtain the historical point cloud model with the highest similarity and the corresponding matching point pair; Based on the matching point pairs, a transformation matrix between the point cloud model and the historical point cloud model is calculated.

Images