A method and system for determining glue filling information based on laser detection and visual recognition

Abstract

The application discloses a kind of based on laser detection and visual identification's information determination method and system of glue filling, the method includes: in the process of mechanical arm gluing, real-time acquisition mechanical arm position data, and through visual identification component and laser detection head of setting at the end of mechanical arm acquisition generation laser detection data;Based on the laser detection data and the mechanical arm position data generate in the glue point cloud image under the coordinate system of mechanical arm, and based on the glue point cloud image generation standard glue model;Based on the glue point cloud image and the standard glue model determines glue height missing information, based on the glue height missing information generation mechanical arm's glue filling path trajectory and the corresponding glue speed of each glue filling position on the glue filling path trajectory.This technical scheme, through laser and visual fusion technology, accurately obtain glue height missing information, realize the accurate quantitative determination of glue filling information, solve the problem of traditional glue filling positioning inaccuracy, quantity control imbalance.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computers, and particularly relates to a glue supplementing information determination method and system based on laser detection and visual recognition. BACKGROUND

[0002] In the manufacturing process of core components such as new energy automobile battery packs and electronic module shells, the uniformity and continuity of the sealing glue directly determine the waterproof and dustproof performance, structural stability and service life safety of the products. However, in actual production, due to factors such as clogging of the glue spraying nozzle, air pressure fluctuation, and trajectory deviation of the mechanical arm, defects such as local glue breakage and glue leakage are difficult to avoid, and precise glue supplementing operation is urgently needed to correct them.

[0003] The traditional defect detection and information determination method before glue supplementing operation mainly relies on manual visual inspection or simple photoelectric sensors. Manual detection is not only time-consuming and labor-intensive, but also easily affected by subjective factors such as the experience and fatigue of the inspector, and cannot accurately obtain the key quantitative information of the glue defect area. Only the approximate range of glue supplementing can be determined by experience. The simple photoelectric sensor can only make a binary judgment of whether it is qualified, and cannot output accurate data support for glue supplementing. Due to the lack of scientific and accurate detection basis for determining glue supplementing information, the subsequent glue supplementing operation has problems such as inaccurate positioning, excessive or insufficient glue supplementing, etc., which ultimately results in uneven glue supplementing quality and cannot meet the stringent requirements of new energy automobile battery packs, electronic module shells and other products on sealing performance. SUMMARY

[0004] The embodiments of the present application provide a glue supplementing information determination method and system based on laser detection and visual recognition, which aims to accurately obtain glue height missing information through laser and visual fusion technology, realize accurate quantitative determination of glue supplementing information, and solve the problems of inaccurate glue supplementing positioning and quantity control imbalance in the traditional method.

[0005] In the first aspect, the embodiments of the present application provide a glue supplementing information determination method based on laser detection and visual recognition, which comprises:

[0006] In the process of glue application by the mechanical arm, the position data of the mechanical arm is acquired in real time, and laser detection data is collected and generated by the visual recognition component and the laser detection head arranged at the end of the mechanical arm;

[0007] Based on the laser detection data and the mechanical arm position data, a glue point cloud image in the mechanical arm coordinate system is generated, and a standard glue model is generated based on the glue point cloud image;

[0008] Based on the glue point cloud image and the standard glue model, glue height missing information is determined, and a glue supplementing path trajectory of the mechanical arm and the glue supplementing speed corresponding to each glue supplementing position on the glue supplementing path trajectory are generated based on the glue height missing information.

[0009] Optionally, the generating a standard gel model based on the gel point cloud image comprises:

[0010] adopting a clustering algorithm to segment the gel point cloud image into a plurality of point cloud clusters, and connecting three-dimensional centroids of the plurality of point cloud clusters to obtain a centerline trajectory;

[0011] carrying out equidistant slicing processing on the gel point cloud image along the centerline trajectory to obtain point cloud parameter information of a plurality of cross-sectional point clouds; wherein the point cloud parameter information comprises point cloud density and point cloud radial distance distribution;

[0012] determining standard cross-sectional information based on the point cloud parameter information of the plurality of cross-sectional point clouds;

[0013] generating a standard gel model based on the standard cross-sectional information and the centerline trajectory.

[0014] Optionally, the determining standard cross-sectional information based on the point cloud parameter information of the plurality of cross-sectional point clouds comprises:

[0015] determining a cross-sectional point cloud with a point cloud density exceeding a preset point cloud density threshold as a candidate cross-sectional point cloud;

[0016] determining a first principal direction and a second principal direction of the candidate cross-sectional point cloud based on the point cloud radial distance distribution of the candidate cross-sectional point cloud, and obtaining a first radial distance in the first principal direction and a second radial distance in the second principal direction;

[0017] determining an average first principal direction based on the first principal direction, an average second principal direction based on the second principal direction, an average first radial distance based on the first radial distance, and an average second radial distance based on the second radial distance to obtain standard cross-sectional information.

[0018] Optionally, the determining gel height missing information based on the gel point cloud image and the standard gel model comprises:

[0019] aligning the standard gel model to the gel point cloud image to obtain a standard point cloud image, comparing the gel point cloud image and the standard point cloud image to obtain a height missing matrix, and calculating a height missing gradient matrix corresponding to the height missing matrix;

[0020] identifying and filtering out a high gradient noise region in the height missing matrix based on the height missing gradient matrix to obtain a target height missing matrix.

[0021] Optionally, the generating the glue-filling path trajectory of the mechanical arm based on the glue height missing information and the glue-filling speed corresponding to each glue-filling position on the glue-filling path trajectory comprises:

[0022] determining a set of connected element coordinates with element values greater than zero in the target height missing matrix as a glue-filling area to be supplemented, and determining a center line trajectory of the glue-filling area to be supplemented as a glue-filling path trajectory;

[0023] calculating a cross-sectional glue-filling amount centered on each glue-filling position on the glue-filling path trajectory based on the target height missing matrix, and calculating an average value of the cross-sectional glue-filling amounts of the glue-filling position and two adjacent glue-filling positions as a target glue-filling amount corresponding to the glue-filling position;

[0024] obtaining a preset mechanical arm end moving speed, and calculating the glue-filling speed corresponding to each glue-filling position on the glue-filling path trajectory based on the preset mechanical arm end moving speed and the target glue-filling amount.

[0025] Optionally, the calculating the glue-filling speed corresponding to each glue-filling position on the glue-filling path trajectory based on the preset mechanical arm end moving speed and the target glue-filling amount comprises:

[0026] determining a basic glue-filling speed corresponding to each glue-filling position on the glue-filling path trajectory based on the preset mechanical arm end moving speed and the target glue-filling amount;

[0027] obtaining glue formula information, and determining a glue viscosity compensation coefficient and a glue solidification compensation coefficient based on the glue formula information;

[0028] calculating the glue-filling speed corresponding to each glue-filling position on the glue-filling path trajectory according to the glue viscosity compensation coefficient, the glue solidification compensation coefficient and the basic glue-filling speed.

[0029] Optionally, after the generating the glue-filling path trajectory of the mechanical arm based on the glue height missing information and the glue-filling speed corresponding to each glue-filling position on the glue-filling path trajectory, the method further comprises:

[0030] generating a mechanical arm motion instruction based on the glue-filling path trajectory, and generating a glue valve control instruction based on the glue-filling speed corresponding to each glue-filling position;

[0031] time-synchronizing and calibrating the mechanical arm motion instruction and the glue valve control instruction;

[0032] controlling the mechanical arm to execute the mechanical arm motion instruction, and controlling a glue flow control valve arranged at the end of the mechanical arm to execute the glue valve control instruction.

[0033] In a second aspect, the embodiments of the present application provide a glue supplement information determination system based on laser detection and visual recognition, which comprises:

[0034] a data acquisition module, configured to acquire real-time mechanical arm position data in a mechanical arm gluing process, and collect laser detection data by a visual recognition component and a laser detection head arranged at the end of the mechanical arm;

[0035] a model generation module, configured to generate a glue point cloud image in a mechanical arm coordinate system based on the laser detection data and the mechanical arm position data, and generate a standard glue model based on the glue point cloud image;

[0036] an information determination module, configured to determine glue height missing information based on the glue point cloud image and the standard glue model, generate a glue supplement path trajectory of the mechanical arm based on the glue height missing information, and determine glue output speeds corresponding to each glue supplement position on the glue supplement path trajectory.

[0037] Optionally, the model generation module is specifically configured to:

[0038] segment the glue point cloud image into a plurality of point cloud clusters by using a clustering algorithm, and connect three-dimensional centroids of the plurality of point cloud clusters to obtain a center line trajectory;

[0039] slice the glue point cloud image at equal intervals along the center line trajectory to obtain point cloud parameter information of a plurality of cross-sectional point clouds; wherein the point cloud parameter information comprises point cloud density and point cloud radial distance distribution;

[0040] determine standard cross-sectional information based on the point cloud parameter information of the plurality of cross-sectional point clouds;

[0041] generate a standard glue model based on the standard cross-sectional information and the center line trajectory.

[0042] Optionally, the model generation module is specifically configured to:

[0043] determine a cross-sectional point cloud with a point cloud density exceeding a preset point cloud density threshold as a candidate cross-sectional point cloud;

[0044] determine a first main direction and a second main direction of the candidate cross-sectional point cloud based on the point cloud radial distance distribution of the candidate cross-sectional point cloud, and obtain a first radial distance in the first main direction and a second radial distance in the second main direction;

[0045] determine an average first main direction based on the first main direction, an average second main direction based on the second main direction, an average first radial distance based on the first radial distance, and an average second radial distance based on the second radial distance, to obtain standard cross-sectional information.

[0046] Optionally, the information determining module is specifically configured to:

[0047] align the standard colloidal model to the colloidal point cloud image to obtain a standard point cloud image, compare the colloidal point cloud image and the standard point cloud image to obtain a height missing matrix, and calculate a height missing gradient matrix corresponding to the height missing matrix;

[0048] identify and filter out a high gradient noise region in the height missing matrix based on the height missing gradient matrix to obtain a target height missing matrix.

[0049] Optionally, the information determining module is specifically configured to:

[0050] determine a set of connected element coordinates with element values greater than zero in the target height missing matrix as a region to be filled with glue, and determine a center line track of the region to be filled with glue as a glue filling path track;

[0051] calculate a cross-sectional glue filling amount centered on each glue filling position on the glue filling path track based on the target height missing matrix, and for each glue filling position, calculate an average value of cross-sectional glue filling amounts of the glue filling position and two adjacent glue filling positions as a target glue output amount corresponding to the glue filling position;

[0052] obtain a preset end-of-arm moving speed, and calculate a glue output speed corresponding to each glue filling position on the glue filling path track based on the preset end-of-arm moving speed and the target glue output amount.

[0053] Optionally, the information determining module is specifically configured to:

[0054] determine a basic glue output speed corresponding to each glue filling position on the glue filling path track based on the preset end-of-arm moving speed and the target glue output amount;

[0055] obtain colloidal formula information, and determine a colloidal viscosity compensation coefficient and a colloidal solidification compensation coefficient based on the colloidal formula information;

[0056] calculate a glue output speed corresponding to each glue filling position on the glue filling path track according to the colloidal viscosity compensation coefficient, the colloidal solidification compensation coefficient, and the basic glue output speed.

[0057] Optionally, the device is further configured to:

[0058] generate a robot motion instruction based on the glue filling path track, and generate a glue valve control instruction based on the glue output speed corresponding to each glue filling position;

[0059] The mechanical arm motion instruction and the glue valve control instruction are time-synchronized and calibrated.

[0060] The mechanical arm executes the mechanical arm motion instruction, and a glue flow control valve arranged at the end of the mechanical arm executes the glue valve control instruction.

[0061] In a third aspect, an electronic device is provided, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, and the program or instruction, when executed by the processor, implements the method of the first aspect.

[0062] In a fourth aspect, a readable storage medium is provided, which stores a program or instruction, and the program or instruction, when executed by a processor, implements the method of the first aspect.

[0063] In the embodiments of the present application, the position data of the mechanical arm is acquired in real time during the glue coating process of the mechanical arm, and laser detection data is collected and generated by a visual recognition component and a laser detection head arranged at the end of the mechanical arm; a glue point cloud image in the coordinate system of the mechanical arm is generated based on the laser detection data and the position data of the mechanical arm, and a standard glue model is generated based on the glue point cloud image; the glue height missing information is determined based on the glue point cloud image and the standard glue model, and the glue coating path trajectory of the mechanical arm and the glue coating speed corresponding to each glue coating position on the glue coating path trajectory are generated based on the glue height missing information. The above-mentioned glue coating information determination method based on laser detection and visual recognition accurately acquires the glue height missing information through laser and visual fusion technology, realizes accurate quantitative determination of the glue coating information, and solves the problems of inaccurate glue coating positioning and unbalanced quantity control in the traditional glue coating. BRIEF DESCRIPTION OF DRAWINGS

[0064] Figure 1 is a flowchart of a glue coating information determination method based on laser detection and visual recognition provided by the embodiments of the present application;

[0065] Figure 2 is a flowchart of another glue coating information determination method based on laser detection and visual recognition provided by the embodiments of the present application;

[0066] Figure 3 is a flowchart of still another glue coating information determination method based on laser detection and visual recognition provided by the embodiments of the present application;

[0067] Figure 4 is an example diagram of a height missing matrix provided by the embodiments of the present application;

[0068] Figure 5is a structural schematic diagram of a system for determining glue-filling information based on laser detection and visual recognition provided by an embodiment of the present application.

[0069] Figure 6 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0070] In order to make the objects, technical solutions and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in combination with the drawings. It can be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application. In addition, it should be noted that, for the convenience of description, only parts related to the present application are shown in the drawings, but not all contents. Before discussing the example embodiments in more detail, it should be mentioned that some example embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The processes can be terminated when the operations are completed, but can also have additional steps not included in the drawings. The processes can correspond to methods, functions, procedures, subroutines, etc.

[0071] The technical solutions in the embodiments of the present application will be described clearly in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art belong to the scope of protection of the present application.

[0072] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of a kind, and are not limited to the number of objects, for example, the first object can be one or more. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / ", generally means that the objects before and after are in an "or" relationship.

[0073] The laser detection and visual recognition-based glue-filling information determination method and system provided by the embodiments of the present application will be described in detail below in combination with the drawings and specific embodiments and application scenarios.

[0074] Firstly, the application is suitable for high-precision gluing scenarios in the field of industrial manufacturing, specifically including but not limited to automobile part assembly, electronic device shell bonding, aerospace component sealing and other scenarios with high requirements for glue body forming precision and thickness consistency, especially suitable for complex curved surface and irregular joint gluing and filling operations. Based on the above use scenarios, it can be understood that the execution subject of the application can be an industrial robot control system, a gluing equipment dedicated controller, or an embedded control unit integrated with a motion control module, a visual processing module and a laser detection module, etc.

[0075] Figure 1 is a flowchart of a glue filling information determination method based on laser detection and visual recognition provided by an embodiment of the application. As shown in Figure 1 , specifically comprising the following steps:

[0076] S101, acquiring mechanical arm position data in real time during the mechanical arm gluing process, and collecting and generating laser detection data through a visual recognition component and a laser detection head arranged at the end of the mechanical arm.

[0077] Wherein, the mechanical arm can be an electromechanical integrated device that can simulate the upper limb movement function of the human body, realize spatial positioning, grabbing, carrying, operation and other actions through a preset program or real-time control signal, and has the core features of multi-degree-of-freedom movement ability, precise positioning accuracy and load capacity adapted to the working scene, and can independently or cooperatively complete various automatic work tasks. The mechanical arm gluing process can be the process of moving the mechanical arm according to the preset path trajectory, continuously or intermittently extruding glue from the end of the glue flow control valve to the gluing area, and forming a preset shape glue layer.

[0078] Wherein, the mechanical arm position data can be the angle data of each joint of the mechanical arm, the three-dimensional coordinates and attitude data of the end effector (glue flow control valve) in the mechanical arm coordinate system, or PLC (Programmable Logic Controller) coordinates.

[0079] In one embodiment, the way to acquire the mechanical arm position data can be to collect the angle data of each joint through the encoder of the mechanical arm, and to obtain the real-time three-dimensional coordinates and attitude data of the end effector through forward kinematics calculation combined with the kinematics model of the mechanical arm.

[0080] Wherein, the visual recognition component can be an optoelectronic sensing device integrated with image acquisition, optical processing, signal conversion and data preprocessing functions, capable of acquiring visual information of the target scene or object, such as industrial CCD (Charge-Coupled Device) camera, 3D structured light camera or binocular vision sensor, etc.

[0081] The mechanical arm end can be a flange plate of the mechanical arm and an end effector mounting seat fixed to the flange plate.

[0082] The laser detection head can be a high-precision detection device integrating laser emission, signal reception, and distance measurement functions, and can obtain three-dimensional profile or distance information of a target surface through laser reflection principle.

[0083] The laser detection data can be three-dimensional space coordinate data of key sampling points on the glue surface in the gluing area in the detection coordinate system, which is obtained based on the laser reflection principle. The detection coordinate system can be a right-handed rectangular coordinate system with the optical center of the visual recognition component as the origin, the X-axis and the Y-axis parallel to the imaging plane of the visual recognition component (the X-axis along the horizontal direction of the imaging plane, and the Y-axis along the vertical direction of the imaging plane), and the Z-axis along the optical axis direction of the visual recognition component (perpendicular to the imaging plane and outward).

[0084] In one embodiment, the laser detection data can be generated by the visual recognition component and the laser detection head arranged at the end of the mechanical arm. The laser detection head can be controlled to emit laser, and the visual recognition component can be controlled to collect two-dimensional image data of the gluing area in real time. A preset visual recognition algorithm can be used to extract laser points from the two-dimensional image data to obtain laser point pixel position information. Based on a pre-constructed calibration relationship, three-dimensional relative coordinates of the laser point pixel position information in the detection coordinate system can be determined as the laser detection data. The preset visual recognition algorithm can include edge detection algorithm, gray change algorithm, morphological processing algorithm, and AI (Artificial Intelligence) segmentation algorithm.

[0085] S102, based on the laser detection data and the mechanical arm position data, a glue point cloud image in the mechanical arm coordinate system is generated, and a standard glue model is generated based on the glue point cloud image.

[0086] The mechanical arm coordinate system can be a right-handed rectangular coordinate system with the center of the mechanical arm base as the origin, the X-axis along the horizontal stretching direction of the mechanical arm, the Y-axis perpendicular to the X-axis in the horizontal plane, and the Z-axis along the vertical direction.

[0087] The glue point cloud image can be a point cloud data set composed of each frame of laser detection data, and each point cloud data contains spatial coordinate information of a position on the glue surface.

[0088] In one embodiment, based on the laser detection data and the mechanical arm position data, the laser detection data can be converted to the mechanical arm coordinate system based on pre-stored hand-eye calibration parameters and mechanical arm position data, that is, the glue point cloud image in the mechanical arm coordinate system.

[0089] The standard colloidal model can be an idealized colloidal three-dimensional model generated by data fitting and morphological optimization based on the spatial distribution characteristics of the effective point cloud data in the colloidal point cloud image. The model eliminates defects such as depressions, protrusions, bubbles and the like on the surface of the colloidal due to glue coating errors and environmental interference, and retains the overall contour morphological characteristics of the colloidal.

[0090] In one embodiment, the standard colloidal model can be generated based on the colloidal point cloud image. The isolated noise points in the colloidal point cloud image can be removed by a radius filtering algorithm. The overall contour trend of the point cloud data can be fitted by a RANSAC algorithm. The two-dimensional path center line and width data of the colloidal can be extracted. The continuous and smooth colloidal contour path can be generated by B-spline curve interpolation based on the two-dimensional path center line and width data. The theoretical thickness of the colloidal can be determined according to the height statistical value of each point cloud data in the colloidal point cloud image. The idealized three-dimensional model with a smooth surface, regular contour and uniform thickness, i.e. the standard colloidal model, can be constructed by combining the smoothed colloidal contour path and the theoretical thickness.

[0091] S103, determining the colloidal height missing information based on the colloidal point cloud image and the standard colloidal model, generating the glue filling path trajectory of the mechanical arm and the glue filling speed corresponding to each glue filling position on the glue filling path trajectory based on the colloidal height missing information.

[0092] The height missing information can be the difference information between the actual height of each point cloud data in the colloidal point cloud image and the preset height of the corresponding position in the standard colloidal model.

[0093] In one embodiment, the colloidal height missing information can be determined based on the colloidal point cloud image and the standard colloidal model. The colloidal point cloud image and the standard colloidal model can be compared point by point. The height difference of each corresponding point can be calculated. The points with positive height difference can be marked as missing points. The adjacent missing points can be clustered into missing areas by a region growing algorithm. The boundary coordinates, maximum missing depth, average missing depth and missing area of each missing area can be determined. Finally, the height missing information including the position, shape and depth distribution of the missing area can be generated.

[0094] The glue filling path trajectory of the mechanical arm can be a glue filling movement path of the end of the mechanical arm planned based on the height missing information.

[0095] The glue filling speed corresponding to each glue filling position on the glue filling path trajectory can be the colloidal extrusion speed calculated based on the height missing depth of each glue filling position on the glue filling path trajectory.

[0096] In one embodiment, the way of generating the glue filling path trajectory of the mechanical arm based on the glue height missing information and the glue output speed corresponding to each glue filling position on the glue filling path trajectory can adopt the following method: generating a smooth glue filling path trajectory by using B-spline curve fitting according to the boundary coordinates of the missing area, calculating the required glue output amount of each glue filling position by using a fluid mechanics formula according to the missing depth of each glue filling position on the glue filling path trajectory and the viscosity parameter of the glue, and then calculating the glue output speed corresponding to each glue filling position based on the glue filling path moving speed and the required glue output amount.

[0097] Optionally, after the glue filling path trajectory of the mechanical arm is generated based on the glue height missing information and the glue output speed corresponding to each glue filling position on the glue filling path trajectory, the method further comprises the following steps of:

[0098] generating a mechanical arm motion instruction based on the glue filling path trajectory, and generating a glue valve control instruction based on the glue output speed corresponding to each glue filling position;

[0099] time-synchronizing and calibrating the mechanical arm motion instruction and the glue valve control instruction;

[0100] controlling the mechanical arm to execute the mechanical arm motion instruction, and controlling the glue flow control valve arranged at the end of the mechanical arm to execute the glue valve control instruction.

[0101] In one embodiment, the mechanical arm motion instruction can be a standardized instruction set containing the motion angle, motion speed, acceleration and motion mode of each joint of the mechanical arm corresponding to the glue filling path trajectory.

[0102] In one embodiment, the way of generating the mechanical arm motion instruction based on the glue filling path trajectory can adopt the following method: calculating the target angle sequence of each joint of the mechanical arm based on the three-dimensional coordinate point sequence of the glue filling path trajectory by using inverse kinematics, generating the motion speed curve and acceleration curve of each joint of the mechanical arm by using trapezoidal velocity planning algorithm in combination with the preset motion constraint parameter, encoding the target angle, speed and acceleration data in the instruction format supported by the mechanical arm controller to form a complete mechanical arm motion instruction.

[0103] In one embodiment, the glue valve control instruction can be a control signal containing the opening timing, closing timing and valve opening parameter corresponding to each glue filling position of the glue flow control valve.

[0104] In one embodiment, the way of generating the glue valve control instruction based on the glue output speed corresponding to each glue filling position can adopt the following method: establishing a calibration mapping table of the glue output speed and the valve opening parameter in advance, querying the calibration mapping table to obtain the corresponding valve opening parameter according to the glue output speed corresponding to each glue filling position, thereby obtaining a valve opening parameter sequence, and encoding the valve opening parameter sequence in the instruction format supported by the driver of the glue flow control valve to form a complete glue valve control instruction.

[0105] In an embodiment, the time synchronization calibration of the robot motion instruction and the glue valve control instruction can be performed by setting a timestamp for the glue valve control instruction based on the arrival time of each glue supplementing position in the robot motion instruction.

[0106] The glue flow control valve can be an electromagnetic proportional flow valve or a servo control flow valve with high-speed response characteristics.

[0107] In an embodiment, the control of the robot to execute the robot motion instruction and the control of the glue flow control valve arranged at the end of the robot to execute the glue valve control instruction can be performed by sending the robot motion instruction to the robot controller in real time through the EtherCAT or Profinet industrial bus, driving the joint servo motor to move according to the instruction, and feeding back the motion state in real time, and sending the glue valve control instruction to the driver of the glue flow control valve through the CAN bus, and adjusting the valve opening according to the instruction.

[0108] The advantages of this scheme are that the dynamic matching of the glue supplementing path movement and the glue discharging speed is achieved through the accurate generation and time synchronization calibration of the robot motion instruction and the glue valve control instruction, and the problems of glue supplementing position deviation, insufficient or excessive glue discharging, etc. are avoided.

[0109] In the embodiment of the present application, the robot position data is acquired in real time during the robot gluing process, and the laser detection data is collected and generated by the visual recognition component and the laser detection head arranged at the end of the robot; the glue point cloud image in the robot coordinate system is generated based on the laser detection data and the robot position data, and the standard glue model is generated based on the glue point cloud image; the glue height missing information is determined based on the glue point cloud image and the standard glue model, and the glue supplementing path trajectory of the robot and the glue discharging speed corresponding to each glue supplementing position on the glue supplementing path trajectory are generated based on the glue height missing information. The glue supplementing information determination method based on laser detection and visual recognition accurately acquires the glue height missing information through laser and visual fusion technology, accurately quantifies the glue supplementing information, and solves the problems of inaccurate glue supplementing positioning and unbalanced quantity control in the prior art.

[0110] Figure 2 is another flowchart of a glue supplementing information determination method based on laser detection and visual recognition provided by the embodiment of the present application. As shown in Figure 2 , the method specifically includes the following steps:

[0111] S201, acquiring the robot position data in real time during the robot gluing process, and collecting and generating the laser detection data by the visual recognition component and the laser detection head arranged at the end of the robot.

[0112] S202, generate a colloidal point cloud image in a mechanical arm coordinate system based on the laser detection data and the mechanical arm position data.

[0113] S203, segment the colloidal point cloud image into a plurality of point cloud clusters using a clustering algorithm, and connect three-dimensional centroids of the plurality of point cloud clusters to obtain a centerline trajectory.

[0114] The clustering algorithm can be a density-based spatial clustering algorithm (DBSCAN), a K-Means clustering algorithm, or a Euclidean clustering algorithm. The core of the clustering algorithm is to group point cloud data in the colloidal point cloud image according to spatial distance and density distribution characteristics of the point cloud data, so as to effectively segment the point cloud.

[0115] The point cloud cluster can be a set of point cloud data adjacent in space obtained by the clustering algorithm from the colloidal point cloud image. Each point cloud cluster corresponds to local point cloud data of a continuous region of the colloidal surface, and can reflect local characteristics of the colloidal.

[0116] In an embodiment, the colloidal point cloud image can be segmented into a plurality of point cloud clusters using a clustering algorithm. A clustering radius threshold and a minimum point cloud quantity threshold can be set first. All point cloud data in the colloidal point cloud image can be traversed. Any unclassified point can be taken as a core. Adjacent points with a distance less than the clustering radius threshold from the core point can be included in the same candidate cluster. When the number of point cloud data in the candidate cluster reaches the minimum point cloud quantity threshold, an effective point cloud cluster is formed. The above process can be repeated until all point cloud data is classified or labeled as an isolated noise point (isolated noise points are directly removed). Finally, a plurality of non-overlapping point cloud clusters are obtained.

[0117] The three-dimensional centroid of the point cloud cluster can be a spatial point corresponding to an arithmetic mean of three-dimensional coordinates of all point cloud data in a single point cloud cluster.

[0118] The centerline trajectory can be a continuous spatial curve formed by connecting three-dimensional centroids of all point cloud clusters. The shape of the centerline trajectory is consistent with the overall extension path of the colloidal, and can accurately reflect the core trend of the colloidal, thereby providing a reference path for subsequent slicing processing.

[0119] In an embodiment, the three-dimensional centroids of the plurality of point cloud clusters can be connected to obtain the centerline trajectory. The three-dimensional centroids of all point cloud clusters can be sorted in a time sequence order of the mechanical arm gluing or a spatial distribution order of the point cloud clusters. A B-spline curve interpolation algorithm can be used to smooth and fit the sorted three-dimensional centroid points to generate a continuous and inflection-free spatial curve. The spatial curve is the centerline trajectory.

[0120] S204, slice the colloidal point cloud image at equal intervals along the center line trajectory to obtain point cloud parameter information of a plurality of cross-sectional point clouds; wherein the point cloud parameter information comprises point cloud density and point cloud radial distance distribution.

[0121] The cross-sectional point cloud can be a colloidal point cloud segment obtained by slicing the colloidal point cloud along the center line trajectory at a predetermined equal interval (for example, 1 mm) in a tangential direction perpendicular to the center line trajectory. The point cloud parameter information of the cross-sectional point cloud can be quantitative data for representing the distribution characteristics, density, and contour size of the cross-sectional point cloud, and can comprise point cloud density and point cloud radial distance distribution. Specifically, the point cloud density can be the number of point cloud data in the cross-sectional point cloud per unit area, and the point cloud radial distance distribution can be statistical distribution data of the radial distance of each point cloud data in the cross-sectional point cloud to the corresponding cross-sectional center of the center line trajectory.

[0122] In an embodiment, the method of slicing the colloidal point cloud image at equal intervals along the center line trajectory to obtain point cloud parameter information of a plurality of cross-sectional point clouds can comprise the following steps: taking a slice reference point every predetermined distance along the extension direction of the center line trajectory, constructing a virtual slice plane perpendicular to the tangential direction of the center line trajectory with the slice reference point as the center, and screening out point cloud data in the colloidal point cloud image that falls within the virtual slice plane to form a cross-sectional point cloud. For each cross-sectional point cloud, the point cloud density and the point cloud radial distance distribution are calculated to finally obtain the point cloud parameter information of each cross-sectional point cloud.

[0123] S205, determining standard cross-sectional information based on the point cloud parameter information of the plurality of cross-sectional point clouds.

[0124] The standard cross-sectional information can be a set of idealized cross-sectional parameters obtained by statistical analysis and optimization based on the point cloud parameter information of all cross-sectional point clouds.

[0125] In an embodiment, the method of determining standard cross-sectional information based on the point cloud parameter information of the plurality of cross-sectional point clouds can comprise the following steps: first, removing abnormal cross-sectional point clouds, then taking the arithmetic mean of the point cloud densities of the remaining effective cross-sectional point clouds as the standard point cloud density, and taking the median of the average radial distances of the effective cross-sectional point clouds as the standard average radial distance, to obtain the standard cross-sectional information.

[0126] Optionally, the method of determining standard cross-sectional information based on the point cloud parameter information of the plurality of cross-sectional point clouds comprises:

[0127] determining the cross-sectional point cloud whose point cloud density exceeds a predetermined point cloud density threshold as a candidate cross-sectional point cloud;

[0128] determine a first principal direction and a second principal direction of the candidate cross-section point cloud based on a point cloud radial distance distribution of the candidate cross-section point cloud, and obtain a first radial distance in the first principal direction and a second radial distance in the second principal direction;

[0129] determine an average first principal direction based on the first principal direction, an average second principal direction based on the second principal direction, an average first radial distance based on the first radial distance, and an average second radial distance based on the second radial distance, to obtain standard cross-section information.

[0130] The preset point cloud density threshold can be a point cloud density value used to screen out cross-section point clouds with sufficient data collection and capable of truly reflecting the cross-section characteristics of the colloid.

[0131] The point cloud density of the cross-section point cloud exceeding the preset point cloud density threshold indicates that the point cloud data distribution of the cross-section point cloud is dense and can comprehensively cover the profile range of the cross-section of the colloid, and the reflected cross-section characteristics have high authenticity and reliability. Therefore, the cross-section point cloud with the point cloud density exceeding the preset point cloud density threshold can be determined as the candidate cross-section point cloud.

[0132] The first principal direction can be a direction in which the point cloud radial distance distribution of the candidate cross-section point cloud is most dispersed (i.e., the long axis direction of the cross-section profile), and the second principal direction can be a direction in which the point cloud radial distance distribution is less dispersed (i.e., the short axis direction of the cross-section profile).

[0133] In an embodiment, the manner of determining the first principal direction and the second principal direction of the candidate cross-section point cloud based on the point cloud radial distance distribution of the candidate cross-section point cloud can include the following steps: constructing a covariance matrix of radial vectors by calculating a set of radial vectors of all point cloud data in the candidate cross-section point cloud to the center of the cross-section, performing eigenvalue decomposition on the covariance matrix, obtaining the eigenvectors corresponding to the two largest eigenvalues, and determining the eigenvectors corresponding to the largest eigenvalues as the first principal direction and the eigenvectors corresponding to the second largest eigenvalues as the second principal direction.

[0134] The first radial distance can be the distance from the center of the cross-section to the farthest point cloud data in the first principal direction of the candidate cross-section point cloud, and the second radial distance can be the distance from the center of the cross-section to the farthest point cloud data in the second principal direction of the candidate cross-section point cloud.

[0135] In an embodiment, the manner of obtaining the first radial distance in the first main direction and the second radial distance in the second main direction can adopt constructing rays along the first main direction and the second main direction respectively, traversing all point cloud data in the candidate cross-section point cloud, calculating the projection distance of each point cloud data in the corresponding main direction (i.e. the component of the radial distance from the point to the center of the cross-section in the main direction), taking the maximum value of the projection distance in the first main direction as the first radial distance, and taking the maximum value of the projection distance in the second main direction as the second radial distance.

[0136] wherein the average first main direction can be a unified direction vector obtained by vector averaging the first main directions of all candidate cross-section point clouds, and the average second main direction can be a unified direction vector obtained by vector averaging the second main directions of all candidate cross-section point clouds.

[0137] In an embodiment, the manner of determining the average first main direction based on the first main direction and determining the average second main direction based on the second main direction can adopt normalizing the first main direction vector and the second main direction vector of each candidate cross-section point cloud (converting to unit vectors), then calculating the arithmetic mean of all normalized first main direction vectors to obtain the average first main direction vector, calculating the arithmetic mean of all normalized second main direction vectors to obtain the average second main direction vector, and finally normalizing the average first main direction vector and the average second main direction vector to determine the final average first main direction and the average second main direction.

[0138] wherein the average first radial distance can be a value obtained by statistically averaging the first radial distances of all candidate cross-section point clouds, and the average second radial distance can be a value obtained by statistically averaging the second radial distances of all candidate cross-section point clouds.

[0139] In an embodiment, the manner of determining the average first radial distance based on the first radial distance and determining the average second radial distance based on the second radial distance can adopt removing outliers of the first radial distance and the second radial distance in the candidate cross-section point cloud, then calculating the arithmetic mean of the remaining valid first radial distances to obtain the average first radial distance, and calculating the arithmetic mean of the remaining valid second radial distances to obtain the average second radial distance.

[0140] The advantage of this arrangement is that it is suitable for adapting to colloids of various cross-section types such as circles and ellipses, significantly improves the accuracy and adaptability of the standard cross-section information, and lays a reliable foundation for the generation of the subsequent standard colloid model.

[0141] S206, generating a standard colloid model based on the standard cross-section information and the centerline trajectory.

[0142] In one embodiment, the method of generating a standard colloidal model based on standard cross-sectional information and centerline trajectory can be achieved by generating a continuous cylindrical structure (wherein the surface subdivision factor of the cylindrical structure is equal to the standard point cloud density) along the extension direction of the centerline trajectory according to the standard average radial distance in the standard cross-sectional information, i.e., a standard colloidal model.

[0143] S207, Based on the colloidal point cloud image and the standard colloidal model, determine the colloidal height missing information, and based on the colloidal height missing information, generate the glue replenishment path trajectory of the robotic arm and the glue dispensing speed corresponding to each glue replenishment position on the glue replenishment path trajectory.

[0144] The advantage of this approach is that it can accurately capture the actual path and cross-sectional features of the colloid. The standard colloid model generated based on these features is closer to the actual shape of the colloid, avoiding the problem of excessive deviation between the preset model and the actual coating trajectory.

[0145] Figure 3 This is a flowchart illustrating another method for determining adhesive filling information based on laser detection and visual recognition provided in this application. Figure 3 As shown, the specific steps include the following:

[0146] S301 acquires the position data of the robotic arm in real time during the adhesive application process and generates laser detection data by using a vision recognition component and a laser detection head set at the end of the robotic arm.

[0147] S302, Based on the laser detection data and the robotic arm position data, generate a colloidal point cloud image in the robotic arm coordinate system, and generate a standard colloidal model based on the colloidal point cloud image.

[0148] S303, Align the standard colloidal model to the colloidal point cloud image to obtain a standard point cloud image, compare the colloidal point cloud image and the standard point cloud image to obtain a height missing matrix, and calculate the height missing gradient matrix corresponding to the height missing matrix.

[0149] The standard point cloud image can be the set of three-dimensional coordinates of a theoretically perfect colloidal surface in the same coordinate system, obtained by spatially registering an ideal standard colloidal model with an actual colloidal point cloud image.

[0150] In one embodiment, the standard colloidal model is aligned to the colloidal point cloud image to obtain the standard point cloud image. This can be achieved by using an iterative nearest-point algorithm, which minimizes the overall distance between the surface of the standard colloidal model and the colloidal point cloud image to complete the spatial registration of the two 3D models.

[0151] The height missing matrix can be a two-dimensional data grid, wherein each grid point stores an absolute value of a height difference of the colloidal point cloud image relative to the standard point cloud image in the normal vector direction.

[0152] In an embodiment, a manner of obtaining the height missing matrix by comparing the colloidal point cloud image and the standard point cloud image can include: for each point in the colloidal point cloud image, searching for a corresponding nearest point in the standard point cloud image, and calculating a distance between the two points in the normal vector direction; and mapping all distance values to a two-dimensional plane to form the height missing matrix.

[0153] Figure 4 FIG. 1 is an example of a height missing matrix provided by an embodiment of the present application. As shown in FIG. 1, the height missing matrix is in the form of a grayscale image, wherein a darker color represents a more serious height missing of the colloidal at the region, and a lighter color represents a colloidal accumulation in line with or exceeding the standard. Figure 4

[0154] The height missing gradient matrix can be a matrix obtained by performing spatial gradient calculation on the height missing matrix, and is used to represent a height change rate and direction of each point in the height missing matrix in a neighborhood of the point.

[0155] In an embodiment, a manner of calculating the height missing gradient matrix corresponding to the height missing matrix can include: performing two-dimensional convolution operation on the height missing matrix by using an image gradient operator such as Sobel or Prewitt, to obtain gradient components in X and Y directions, and then calculating a total gradient amplitude to obtain the height missing gradient matrix.

[0156] S304, identifying and filtering out a high-gradient noise region in the height missing matrix based on the height missing gradient matrix, to obtain a target height missing matrix.

[0157] The high-gradient noise region in the height missing matrix can be caused by factors such as point cloud collection noise, colloidal surface reflection, or attached bubbles, and is represented as an isolated, boundary-abnormal steep mutation region in the height missing matrix.

[0158] The target height missing matrix can be a clean height missing matrix that only contains real and continuous colloidal missing regions after filtering out the high-gradient noise region.

[0159] ​In one embodiment, the way of identifying and filtering out the high gradient noise area in the height missing matrix based on the height missing gradient matrix to obtain the target height missing matrix can be that a gradient amplitude threshold is set, the area in the height missing gradient matrix with a gradient amplitude exceeding the gradient amplitude threshold is marked as a mask, and the mask is used to filter the original height missing matrix, the values of these areas are set to zero or filled with the average value of the neighborhood to obtain the target height missing matrix.

[0160] S305, generating a glue filling path trajectory of the mechanical arm and a glue output speed corresponding to each glue filling position on the glue filling path trajectory based on the height missing information of the glue.

[0161] Optionally, the generating of the glue filling path trajectory of the mechanical arm and the glue output speed corresponding to each glue filling position on the glue filling path trajectory based on the height missing information of the glue includes:

[0162] determining a connected element coordinate set with an element value greater than zero in the target height missing matrix as a glue filling area, and determining a center line trajectory of the glue filling area as the glue filling path trajectory;

[0163] calculating a cross-sectional glue filling amount centered on each glue filling position on the glue filling path trajectory based on the target height missing matrix, and calculating an average value of the cross-sectional glue filling amounts of the glue filling position and two adjacent glue filling positions as a target glue output amount corresponding to the glue filling position;

[0164] obtaining a preset mechanical arm end moving speed, and calculating the glue output speed corresponding to each glue filling position on the glue filling path trajectory based on the preset mechanical arm end moving speed and the target glue output amount.

[0165] wherein an element value greater than zero in the target height missing matrix indicates that the glue height at the coordinate point is lower than the standard height, and there is height missing.

[0166] wherein the connected element coordinate set can be a region composed of adjacent pixels or data points with element values greater than zero, which is identified by a region growing method or a morphological operation; and the glue filling area can be one or more continuous spatial regions determined by connectivity analysis and requiring glue filling operation.

[0167] In one embodiment, the way of determining the center line trajectory of the glue filling area as the glue filling path trajectory can be that a skeletonization algorithm or a medial axis transformation algorithm is used to extract a center skeleton line of the glue filling area as the center line trajectory.

[0168] The cross-sectional glue-filling amount centered on each glue-filling position on the glue-filling path trajectory can be the total missing amount obtained by integrating or summing the element values of the corresponding positions in the target height missing matrix on a cross section perpendicular to the glue-filling path trajectory.

[0169] In an embodiment, the manner of calculating the cross-sectional glue-filling amount centered on each glue-filling position on the glue-filling path trajectory based on the target height missing matrix can be to accumulate the missing values of all corresponding points in the matrix within a preset width cross-sectional strip centered on each glue-filling position on the glue-filling path trajectory.

[0170] The target glue-filling amount corresponding to the glue-filling position can be the glue volume planned to be allocated to compensate for the height missing of the position and its adjacent area.

[0171] The adjacent glue-filling positions can be the upstream glue-filling position and the downstream glue-filling position closest to the current glue-filling position on the glue-filling path trajectory.

[0172] In an embodiment, for each glue-filling position, the manner of calculating the average value of the cross-sectional glue-filling amounts of the glue-filling position and the two adjacent glue-filling positions as the target glue-filling amount corresponding to the glue-filling position can be to add the cross-sectional glue-filling amounts of the glue-filling position and the two adjacent glue-filling positions, and divide the sum by 3 to obtain the target glue-filling amount corresponding to the glue-filling position.

[0173] The preset mechanical arm end moving speed can be a constant moving speed of the end of the mechanical arm when performing the glue-filling operation, which is preset according to process requirements.

[0174] In an embodiment, the manner of obtaining the preset mechanical arm end moving speed can be to directly read a pre-constructed glue coating process parameter database.

[0175] In an embodiment, the manner of calculating the glue-filling speed corresponding to each glue-filling position on the glue-filling path trajectory based on the preset mechanical arm end moving speed and the target glue-filling amount can be to divide the target glue-filling amount corresponding to each glue-filling position on the glue-filling path trajectory by the preset mechanical arm end moving speed to obtain the glue-filling speed corresponding to each glue-filling position on the glue-filling path trajectory.

[0176] Optionally, the calculating the glue-filling speed corresponding to each glue-filling position on the glue-filling path trajectory based on the preset mechanical arm end moving speed and the target glue-filling amount comprises:

[0177] determining a basic glue-filling speed corresponding to each glue-filling position on the glue-filling path trajectory based on the preset mechanical arm end moving speed and the target glue-filling amount;

[0178] obtaining colloid formula information, and determining a colloid viscosity compensation coefficient and a colloid solidification compensation coefficient based on the colloid formula information;

[0179] calculating a glue output speed corresponding to each glue filling position on the glue filling path trajectory according to the colloid viscosity compensation coefficient, the colloid solidification compensation coefficient, and the base glue output speed.

[0180] The base glue output speed can be a theoretical glue output speed calculated according to an end moving speed of a mechanical arm under the premise that the influence of colloid self characteristics is not considered, to meet a target glue output requirement.

[0181] In an embodiment, the manner of determining the base glue output speed corresponding to each glue filling position on the glue filling path trajectory based on the preset end moving speed of the mechanical arm and the target glue output can adopt dividing the target glue output corresponding to each glue filling position on the glue filling path trajectory by the preset end moving speed of the mechanical arm to obtain the base glue output speed corresponding to each glue filling position on the glue filling path trajectory.

[0182] The colloid formula information can be a parameter set for identifying colloid types, component proportions, and key physical and chemical properties, such as colloid models, viscosity grades, and solidification rates.

[0183] In an embodiment, the manner of obtaining the colloid formula information can adopt directly reading a pre-constructed glue application process parameter database.

[0184] The colloid viscosity compensation coefficient can be a multiplier factor for modifying the glue output speed to compensate for the influence of flow resistance caused by colloid viscosity; and the colloid solidification compensation coefficient can be a multiplier factor for modifying the glue output speed to compensate for the influence of colloid solidification characteristics (such as thixotropy and solidification shrinkage).

[0185] In an embodiment, the manner of determining the colloid viscosity compensation coefficient and the colloid solidification compensation coefficient based on the colloid formula information can adopt querying a pre-constructed mapping table of colloid formula information-colloid viscosity compensation coefficient-colloid solidification compensation coefficient, and directly obtaining the corresponding colloid viscosity compensation coefficient and colloid solidification compensation coefficient according to the current colloid formula information.

[0186] In an embodiment, the manner of calculating the glue output speed corresponding to each glue filling position on the glue filling path trajectory according to the colloid viscosity compensation coefficient, the colloid solidification compensation coefficient, and the base glue output speed can adopt multiplying the colloid viscosity compensation coefficient, the colloid solidification compensation coefficient, and the base glue output speed to obtain the glue output speed corresponding to each glue filling position on the glue filling path trajectory.

[0187] The advantage of this scheme is that by introducing the colloid formula characteristics to dynamically compensate the glue-out process, the accuracy and adaptability of glue-out control can be significantly improved, effectively overcoming the process fluctuations caused by colloid viscosity changes and solidification behavior.

[0188] The advantage of this scheme is that by dynamically adjusting the glue-out speed, precise and adaptive automatic glue compensation is achieved.

[0189] The advantage of this scheme is that by introducing gradient analysis to intelligently filter noise, the real colloid missing area can be accurately identified, and the glue compensation path trajectory and glue-out speed generated on this basis can better help to achieve quantitative and adaptive compensation for the missing glue area, avoiding false compensation or missing compensation caused by noise interference, and effectively ensuring the product glue quality.

[0190] Figure 5 is a structure schematic diagram of a glue compensation information determination system based on laser detection and visual recognition provided by the embodiment of the present application. As shown in Figure 5 , the system comprises:

[0191] The data acquisition module 510 is configured to acquire the mechanical arm position data in real time during the mechanical arm glue application process, and collect and generate laser detection data through the visual recognition component and the laser detection head arranged at the end of the mechanical arm.

[0192] The model generation module 520 is configured to generate a colloid point cloud image in the mechanical arm coordinate system based on the laser detection data and the mechanical arm position data, and generate a standard colloid model based on the colloid point cloud image.

[0193] The information determination module 530 is configured to determine colloid height missing information based on the colloid point cloud image and the standard colloid model, generate a glue compensation path trajectory of the mechanical arm based on the colloid height missing information, and generate a glue-out speed corresponding to each glue compensation position on the glue compensation path trajectory.

[0194] Optionally, the model generation module 520 is specifically configured to:

[0195] The clustering algorithm is used to segment the colloid point cloud image into a plurality of point cloud clusters, and the three-dimensional centroids of the plurality of point cloud clusters are connected to obtain a center line trajectory;

[0196] The colloid point cloud image is processed by equal-interval slicing along the center line trajectory to obtain point cloud parameter information of a plurality of cross-sectional point clouds; wherein the point cloud parameter information includes point cloud density and point cloud radial distance distribution;

[0197] The standard cross-sectional information is determined based on the point cloud parameter information of the plurality of cross-sectional point clouds.

[0198] generate a standard colloid model based on the standard cross-section information and the center line trajectory.

[0199] Optionally, the model generation module 520 is specifically configured to:

[0200] determine the cross-section point cloud whose point cloud density exceeds a preset point cloud density threshold as a candidate cross-section point cloud;

[0201] determine a first principal direction and a second principal direction of the candidate cross-section point cloud based on a point cloud radial distance distribution of the candidate cross-section point cloud, and obtain a first radial distance in the first principal direction and a second radial distance in the second principal direction;

[0202] determine an average first principal direction based on the first principal direction, an average second principal direction based on the second principal direction, an average first radial distance based on the first radial distance, and an average second radial distance based on the second radial distance, to obtain standard cross-section information.

[0203] Optionally, the information determination module 530 is specifically configured to:

[0204] align the standard colloid model to the colloid point cloud image to obtain a standard point cloud image, compare the colloid point cloud image and the standard point cloud image to obtain a height loss matrix, and calculate a height loss gradient matrix corresponding to the height loss matrix;

[0205] identify and filter out a high gradient noise area in the height loss matrix based on the height loss gradient matrix, to obtain a target height loss matrix.

[0206] Optionally, the information determination module 530 is specifically configured to:

[0207] determine a connected element coordinate set with an element value greater than zero in the target height loss matrix as a to-be-filled glue area, and determine a center line trajectory of the to-be-filled glue area as a glue filling path trajectory;

[0208] calculate a cross-section glue filling amount centered on each glue filling position on the glue filling path trajectory based on the target height loss matrix, and for each glue filling position, calculate an average value of cross-section glue filling amounts of the glue filling position and two adjacent glue filling positions as a target glue output amount corresponding to the glue filling position;

[0209] obtain a preset mechanical arm end moving speed, and calculate a glue output speed corresponding to each glue filling position on the glue filling path trajectory based on the preset mechanical arm end moving speed and the target glue output amount.

[0210] Optionally, the information determination module 530 is specifically configured to:

[0211] determine a basic glue output speed corresponding to each glue filling position on the glue filling path trajectory based on the preset mechanical arm end moving speed and the target glue output amount;

[0212] obtain glue formula information, and determine a glue viscosity compensation coefficient and a glue solidification compensation coefficient based on the glue formula information;

[0213] calculate the glue output speed corresponding to each glue filling position on the glue filling path trajectory based on the glue viscosity compensation coefficient, the glue solidification compensation coefficient, and the basic glue output speed.

[0214] Optionally, the device is further used for:

[0215] generating a mechanical arm motion instruction based on the glue filling path trajectory, and generating a glue valve control instruction based on the glue output speed corresponding to each glue filling position;

[0216] time-synchronize and calibrate the mechanical arm motion instruction and the glue valve control instruction;

[0217] control the mechanical arm to execute the mechanical arm motion instruction, and control a glue flow control valve arranged at the mechanical arm end to execute the glue valve control instruction.

[0218] In the embodiments of the present application, the data acquisition module is used to acquire mechanical arm position data in real time during the mechanical arm glue coating process, and laser detection data is collected and generated through a visual recognition component and a laser detection head arranged at the mechanical arm end; the model generation module is used to generate a glue point cloud image in the mechanical arm coordinate system based on the laser detection data and the mechanical arm position data, and generate a standard glue model based on the glue point cloud image; the information determination module is used to determine glue height missing information based on the glue point cloud image and the standard glue model, generate a glue filling path trajectory of the mechanical arm and a glue output speed corresponding to each glue filling position on the glue filling path trajectory based on the glue height missing information. The above glue filling information determination system based on laser detection and visual recognition accurately acquires glue height missing information through laser and visual fusion technology, realizes accurate quantitative determination of glue filling information, and solves the problems of inaccurate glue filling positioning and unbalanced quantity control in the prior art.

[0219] The laser detection and visual recognition based glue filling information determination system in the embodiments of the present application can be a system, or a component, integrated circuit or chip in a terminal. The system can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, tablet computer, notebook computer, palm computer, vehicle-mounted electronic device, wearable device, ultra-mobile personal computer (UMPC), netbook or personal digital assistant (PDA), etc., and the non-mobile electronic device can be a server, network attached storage (NAS), personal computer (PC), television (TV), teller machine or self-service machine, etc., and the embodiments of the present application are not limited in this regard.

[0220] The laser detection and visual recognition based glue filling information determination system in the embodiments of the present application can be a system with an operating system. The operating system can be an Android operating system, an IOS operating system or other possible operating system, and the embodiments of the present application are not limited in this regard.

[0221] The laser detection and visual recognition based glue filling information determination system provided in the embodiments of the present application can implement the processes implemented by the above-mentioned embodiments, and thus repeated description is omitted here.

[0222] Figure 6 FIG. 1 is a structural schematic diagram of an electronic device provided in an embodiment of the present application. As shown in FIG. 1, the electronic device 100 provided in the embodiment of the present application includes a processor 101, a memory 102, and a program or instruction stored in the memory 102 and executable on the processor 101. Figure 6 The processor 101 executes the program or instruction to implement each process of the above-mentioned laser detection and visual recognition based glue filling information determination method embodiment, and thus achieves the same technical effects. Repeated description is omitted here.

[0223] It should be noted that the electronic device in the embodiments of the present application includes the above-mentioned mobile electronic device and non-mobile electronic device.

[0224] The embodiments of the present application further provide a readable storage medium having a program or instruction stored thereon. The program or instruction is executable on a processor to implement each process of the above-mentioned laser detection and visual recognition based glue filling information determination method embodiment, and thus achieves the same technical effects. Repeated description is omitted here.

[0225] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0226] It should be noted that, in this document, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element preceded by "comprises... a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or system that includes the element. In addition, it should be noted that the scope of the methods and systems of the present embodiments are not limited to performing functions in the order recited in the specification or as may be illustrated in the accompanying drawings. The methods and systems can include additional or fewer steps, or can be performed in a different order than as illustrated or discussed herein. Furthermore, features described in relation to certain examples can be combined in other examples.

[0227] From the above description of the embodiments, it can be clear to those skilled in the art that the above-mentioned embodiment methods can be realized by means of software and necessary general hardware platforms, of course, they can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a plurality of instructions for making a terminal (which can be a mobile phone, computer, server, or network device, etc.) execute the methods described in various embodiments of the present application.

[0228] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above specific embodiments, and the above specific embodiments are only illustrative, not restrictive. Those skilled in the art can make many forms under the inspiration of the present application without departing from the scope of the present application and the protection scope of the claims.

[0229] The above merely describes the preferred embodiments of the present application and the technical principles applied. The present application is not limited to the specific embodiments described herein, and various obvious changes, modifications and replacements made by those skilled in the art without departing from the scope of the present application shall not be excluded. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and more other equivalent embodiments can be included without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A method for determining glue-filling information based on laser detection and visual recognition, characterized in that, The method comprises: Real-time acquisition of mechanical arm position data in the mechanical arm gluing process, and collection of laser detection data generated by a visual recognition component and a laser detection head arranged at the end of the mechanical arm; Generation of a glue point cloud image in the mechanical arm coordinate system based on the laser detection data and the mechanical arm position data; segmentation of the glue point cloud image into multiple point cloud clusters using a clustering algorithm, and connection of three-dimensional centroids of the multiple point cloud clusters to obtain a center line trajectory; equal-interval slicing processing of the glue point cloud image along the center line trajectory to obtain point cloud parameter information of multiple cross-sectional point clouds; wherein the point cloud parameter information comprises point cloud density and point cloud radial distance distribution; determination of standard cross-sectional information based on the point cloud parameter information of the multiple cross-sectional point clouds; and generation of a standard glue model based on the standard cross-sectional information and the center line trajectory; Determination of glue height missing information based on the glue point cloud image and the standard glue model, and generation of a glue filling path trajectory of the mechanical arm and a glue filling speed corresponding to each glue filling position on the glue filling path trajectory based on the glue height missing information. 2.The method of claim 1, wherein, The determination of the standard cross-sectional information based on the point cloud parameter information of the multiple cross-sectional point clouds comprises: Determination of a cross-sectional point cloud with a point cloud density exceeding a preset point cloud density threshold as a candidate cross-sectional point cloud; Determination of a first main direction and a second main direction of the candidate cross-sectional point cloud based on the point cloud radial distance distribution of the candidate cross-sectional point cloud, and acquisition of a first radial distance in the first main direction and a second radial distance in the second main direction; Determination of an average first main direction based on the first main direction, an average second main direction based on the second main direction, an average first radial distance based on the first radial distance, and an average second radial distance based on the second radial distance to obtain standard cross-sectional information. 3.The method of claim 1 or 2, wherein, The determination of the glue height missing information based on the glue point cloud image and the standard glue model comprises: Alignment of the standard glue model to the glue point cloud image to obtain a standard point cloud image, comparison of the glue point cloud image and the standard point cloud image to obtain a height missing matrix, and calculation of a height missing gradient matrix corresponding to the height missing matrix; Identification and filtering of high gradient noise regions in the height missing matrix based on the height missing gradient matrix to obtain a target height missing matrix. 4.The method of claim 3, wherein, The generation of the glue filling path trajectory of the mechanical arm and the glue filling speed corresponding to each glue filling position on the glue filling path trajectory based on the glue height missing information comprises: Determination of a connected element coordinate set with an element value greater than zero in the target height missing matrix as a glue filling area to be filled, and determination of a center line trajectory of the glue filling area to be filled as a glue filling path trajectory; Calculation of cross-sectional glue filling amounts centered on each glue filling position on the glue filling path trajectory based on the target height missing matrix, and calculation of an average value of cross-sectional glue filling amounts of the glue filling position and two adjacent glue filling positions as a target glue filling amount corresponding to the glue filling position for each glue filling position. The preset mechanical arm end moving speed is acquired, and a glue output speed corresponding to each glue filling position on the glue filling path trajectory is calculated based on the preset mechanical arm end moving speed and the target glue output amount.

5. The method for determining glue filling information based on laser detection and visual identification according to claim 4, characterized in that, The calculation of the glue output speed corresponding to each glue filling position on the glue filling path trajectory based on the preset mechanical arm end moving speed and the target glue output amount comprises: determining a basic glue output speed corresponding to each glue filling position on the glue filling path trajectory based on the preset mechanical arm end moving speed and the target glue output amount; acquiring glue formula information, and determining a glue viscosity compensation coefficient and a glue solidification compensation coefficient based on the glue formula information; calculating the glue output speed corresponding to each glue filling position on the glue filling path trajectory based on the glue viscosity compensation coefficient, the glue solidification compensation coefficient and the basic glue output speed. 6.The method of determining glue-filling information based on laser detection and visual identification according to claim 1, wherein, After the generation of the glue filling path trajectory of the mechanical arm and the glue output speed corresponding to each glue filling position on the glue filling path trajectory based on the glue height missing information, the method further comprises: generating a mechanical arm movement instruction based on the glue filling path trajectory, and generating a glue valve control instruction based on the glue output speed corresponding to each glue filling position; time-synchronizing and calibrating the mechanical arm movement instruction and the glue valve control instruction; controlling the mechanical arm to execute the mechanical arm movement instruction, and controlling a glue flow control valve arranged at the mechanical arm end to execute the glue valve control instruction.

7. A system for determining glue-filling information based on laser detection and visual recognition, characterized in that, The system comprises: a data acquisition module configured to acquire mechanical arm position data in real time during a mechanical arm glue coating process, and collect laser detection data by a visual recognition component and a laser detection head arranged at a mechanical arm end; a model generation module configured to: generate a glue point cloud image in a mechanical arm coordinate system based on the laser detection data and the mechanical arm position data; segment the glue point cloud image into a plurality of point cloud clusters by using a clustering algorithm, and connect three-dimensional centroids of the plurality of point cloud clusters to obtain a center line trajectory; perform equidistant slicing processing on the glue point cloud image along the center line trajectory to obtain point cloud parameter information of a plurality of cross-sectional point clouds; wherein the point cloud parameter information comprises point cloud density and point cloud radial distance distribution; determine standard cross-sectional information based on the point cloud parameter information of the plurality of cross-sectional point clouds; and generate a standard glue model based on the standard cross-sectional information and the center line trajectory; an information determination module configured to determine glue height missing information based on the glue point cloud image and the standard glue model, generate a glue filling path trajectory of a mechanical arm and a glue output speed corresponding to each glue filling position on the glue filling path trajectory based on the glue height missing information.

8. An electronic device, comprising: The processor, the memory and the program or instructions stored on the memory and executable on the processor are included, and the program or instructions are executed by the processor to implement the glue filling information determination method based on laser detection and visual recognition in any one of claims 1-6.

9. A readable storage medium, characterized by, The readable storage medium stores programs or instructions, and the programs or instructions are executed by the processor to implement the laser detection and visual recognition based glue supplement information determination method in any one of claims 1-6.

Images