Method and system for determining glue filling information based on laser detection and visual identification

Abstract

The invention discloses a glue filling information determination method and system based on laser detection and visual identification, and the method comprises the steps: obtaining the position data of a mechanical arm in real time in the gluing process of the mechanical arm, and collecting and generating laser detection data through a visual identification assembly and a laser detection head disposed at the tail end of the mechanical arm; generating a colloid point cloud image under a mechanical arm coordinate system based on the laser detection data and the mechanical arm position data, and generating a standard colloid model based on the colloid point cloud image; and based on the colloid point cloud image and the standard colloid model, determining colloid height missing information, and based on the colloid height missing information, generating a glue supplementing path track of the mechanical arm and the glue discharging speed corresponding to each glue supplementing position on the glue supplementing path track. According to the technical scheme, the glue height missing information is accurately obtained through the laser and vision fusion technology, accurate quantitative determination of glue supplementing information is achieved, and the problems that traditional glue supplementing is inaccurate in positioning and unbalanced in quantity control are solved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computers, and particularly relates to a glue filling information determination method and system based on laser detection and visual recognition. BACKGROUND

[0002] In the manufacturing process of core components such as new energy automobile battery packs and electronic module shells, the uniformity and continuity of the sealing glue directly determine the waterproof and dustproof performance, structural stability and service life safety of the products. However, in actual production, due to factors such as clogging of the glue spraying nozzle, air pressure fluctuation, and trajectory deviation of the mechanical arm, defects such as local glue breakage and glue leakage are difficult to avoid, and precise glue filling operations are urgently needed to correct them.

[0003] The traditional defect detection and information determination method before the glue filling operation mainly relies on manual visual inspection or simple photoelectric sensors. Manual detection is not only time-consuming and labor-intensive, but also easily affected by subjective factors such as the experience and fatigue of the inspector, and cannot accurately obtain the key quantitative information of the glue missing area. It can only determine the approximate range of glue filling based on experience. The simple photoelectric sensor can only make a binary judgment of whether it is qualified, and cannot output accurate data support for glue filling. Due to the lack of scientific and accurate detection basis for determining glue filling information, the subsequent glue filling operation has problems such as inaccurate positioning, excessive or insufficient glue filling, etc., which ultimately leads to uneven glue filling quality and cannot meet the stringent requirements of new energy automobile battery packs, electronic module shells and other products on sealing performance. SUMMARY

[0004] The embodiments of the present application provide a glue filling information determination method and system based on laser detection and visual recognition, which aims to accurately obtain glue height missing information through laser and visual fusion technology, realize accurate quantitative determination of glue filling information, and solve the problems of inaccurate glue filling positioning and quantity control imbalance in the traditional glue filling.

[0005] In the first aspect, the embodiments of the present application provide a glue filling information determination method based on laser detection and visual recognition, which comprises: Real-time acquisition of mechanical arm position data during the glue filling process of the mechanical arm, and acquisition of laser detection data by a visual recognition component and a laser detection head arranged at the end of the mechanical arm; Generation of a glue point cloud image in the mechanical arm coordinate system based on the laser detection data and the mechanical arm position data, and generation of a standard glue model based on the glue point cloud image; Determination of glue height missing information based on the glue point cloud image and the standard glue model, generation of a glue filling path trajectory of the mechanical arm and a glue filling speed corresponding to each glue filling position on the glue filling path trajectory based on the glue height missing information.

[0006] Optionally, the generating a standard colloidal model based on the colloidal point cloud image comprises: adopting a clustering algorithm to segment the colloidal point cloud image into a plurality of point cloud clusters, and connecting three-dimensional centroids of the plurality of point cloud clusters to obtain a center line trajectory; carrying out equidistant slicing processing on the colloidal point cloud image along the center line trajectory to obtain point cloud parameter information of a plurality of cross-sectional point clouds; wherein the point cloud parameter information comprises point cloud density and point cloud radial distance distribution; determining standard cross-sectional information based on the point cloud parameter information of the plurality of cross-sectional point clouds; generating a standard colloidal model based on the standard cross-sectional information and the center line trajectory.

[0007] Optionally, the determining standard cross-sectional information based on the point cloud parameter information of the plurality of cross-sectional point clouds comprises: determining a cross-sectional point cloud with a point cloud density exceeding a preset point cloud density threshold as a candidate cross-sectional point cloud; determining a first principal direction and a second principal direction of the candidate cross-sectional point cloud based on the point cloud radial distance distribution of the candidate cross-sectional point cloud, and obtaining a first radial distance in the first principal direction and a second radial distance in the second principal direction; determining an average first principal direction based on the first principal direction, an average second principal direction based on the second principal direction, an average first radial distance based on the first radial distance, and an average second radial distance based on the second radial distance to obtain standard cross-sectional information.

[0008] Optionally, the determining colloidal height missing information based on the colloidal point cloud image and the standard colloidal model comprises: aligning the standard colloidal model to the colloidal point cloud image to obtain a standard point cloud image, comparing the colloidal point cloud image and the standard point cloud image to obtain a height missing matrix, and calculating a height missing gradient matrix corresponding to the height missing matrix; based on the height missing gradient matrix, identifying and filtering out high gradient noise regions in the height missing matrix to obtain a target height missing matrix.

[0009] Optionally, the generating a glue filling path trajectory of a mechanical arm and a glue output speed corresponding to each glue filling position on the glue filling path trajectory based on the colloidal height missing information comprises: determining a connected element coordinate set with an element value greater than zero in the target height missing matrix as a glue filling area to be filled, and determining a center line trajectory of the glue filling area to be filled as a glue filling path trajectory; Based on the target height missing matrix, calculate the cross-sectional glue filling amount centered on each glue filling position on the glue filling path trajectory. For each glue filling position, calculate the average cross-sectional glue filling amount of the glue filling position and two adjacent glue filling positions as the target glue output amount corresponding to the glue filling position. Obtain a preset end-effector movement speed, and calculate the glue dispensing speed corresponding to each glue dispensing position on the glue dispensing path trajectory based on the preset end-effector movement speed and the target glue dispensing amount.

[0010] Optionally, the step of calculating the dispensing speed corresponding to each dispensing position on the dispensing path trajectory based on the preset robotic arm end-effector movement speed and the target dispensing amount includes: Based on the preset end-effector movement speed and the target glue dispensing amount, the basic glue dispensing speed corresponding to each glue dispensing position on the glue dispensing path trajectory is determined. Obtain colloidal formulation information, and determine the colloidal viscosity compensation coefficient and the colloidal curing compensation coefficient based on the colloidal formulation information; The dispensing speed at each dispensing position on the dispensing path trajectory is calculated based on the colloid viscosity compensation coefficient, the colloid curing compensation coefficient, and the basic dispensing speed.

[0011] Optionally, after generating the glue replenishment path trajectory of the robotic arm based on the glue height loss information and the glue dispensing speed corresponding to each glue replenishment position on the glue replenishment path trajectory, the method further includes: Based on the glue application path trajectory, a robotic arm motion command is generated, and based on the glue dispensing speed corresponding to each glue application position, a glue valve control command is generated. The robotic arm motion commands and the glue valve control commands are time-synchronized and calibrated. The robotic arm is controlled to execute the robotic arm movement commands, and the colloid flow control valve located at the end of the robotic arm is controlled to execute the colloid valve control commands.

[0012] Secondly, embodiments of this application provide a system for determining adhesive replenishment information based on laser detection and visual recognition, the system comprising: The data acquisition module is used to acquire the position data of the robotic arm in real time during the adhesive application process, and to collect and generate laser detection data through the vision recognition component and the laser detection head set at the end of the robotic arm. The model generation module is used to generate a colloidal point cloud image in the robotic arm coordinate system based on the laser detection data and the robotic arm position data, and to generate a standard colloidal model based on the colloidal point cloud image. The information determination module is used to determine the missing colloid height information based on the colloid point cloud image and the standard colloid model, and to generate the colloid replenishment path trajectory of the robotic arm and the dispensing speed corresponding to each colloid replenishment position on the colloid replenishment path trajectory based on the missing colloid height information.

[0013] Optionally, the model generation module is specifically used for: The colloidal point cloud image is segmented into multiple point cloud clusters using a clustering algorithm, and the three-dimensional centroids of the multiple point cloud clusters are connected to obtain the centerline trajectory. Following the centerline trajectory, the colloidal point cloud image is sliced ​​at equal intervals to obtain point cloud parameter information for multiple cross-sectional point clouds; wherein, the point cloud parameter information includes point cloud density and point cloud radial distance distribution; Standard cross-sectional information is determined based on the point cloud parameter information of the multiple cross-sectional point clouds; A standard colloidal model is generated based on the standard cross-sectional information and the centerline trajectory.

[0014] Optionally, the model generation module is specifically used for: Cross-sectional point clouds with point cloud density exceeding a preset point cloud density threshold are identified as candidate cross-sectional point clouds. Based on the radial distance distribution of the point cloud of the candidate cross-section, the first principal direction and the second principal direction of the point cloud of the candidate cross-section are determined, and the first radial distance in the first principal direction and the second radial distance in the second principal direction are obtained; The average first principal direction is determined based on the first principal direction, the average second principal direction is determined based on the second principal direction, the average first radial distance is determined based on the first radial distance, and the average second radial distance is determined based on the second radial distance to obtain standard cross-sectional information.

[0015] Optionally, the information determination module is specifically used for: Align the standard colloidal model to the colloidal point cloud image to obtain a standard point cloud image. Compare the colloidal point cloud image and the standard point cloud image to obtain a height missing matrix, and calculate the height missing gradient matrix corresponding to the height missing matrix. Based on the height missing gradient matrix, high gradient noise regions in the height missing matrix are identified and filtered out to obtain the target height missing matrix.

[0016] Optionally, the information determination module is specifically used for: The set of coordinates of connected elements with values ​​greater than zero in the target height missing matrix is ​​determined as the region to be glued, and the centerline trajectory of the region to be glued is determined as the glued path trajectory. Based on the target height missing matrix, calculate the cross-sectional glue filling amount centered on each glue filling position on the glue filling path trajectory. For each glue filling position, calculate the average cross-sectional glue filling amount of the glue filling position and two adjacent glue filling positions as the target glue output amount corresponding to the glue filling position. Obtain a preset end-effector movement speed, and calculate the glue dispensing speed corresponding to each glue dispensing position on the glue dispensing path trajectory based on the preset end-effector movement speed and the target glue dispensing amount.

[0017] Optionally, the information determination module is specifically used for: Based on the preset end-effector movement speed and the target glue dispensing amount, the basic glue dispensing speed corresponding to each glue dispensing position on the glue dispensing path trajectory is determined. Obtain colloidal formulation information, and determine the colloidal viscosity compensation coefficient and the colloidal curing compensation coefficient based on the colloidal formulation information; The dispensing speed at each dispensing position on the dispensing path trajectory is calculated based on the colloid viscosity compensation coefficient, the colloid curing compensation coefficient, and the basic dispensing speed.

[0018] Optionally, the device is also used for: Based on the glue application path trajectory, a robotic arm motion command is generated, and based on the glue dispensing speed corresponding to each glue application position, a glue valve control command is generated. The robotic arm motion commands and the glue valve control commands are time-synchronized and calibrated. The robotic arm is controlled to execute the robotic arm movement commands, and the colloid flow control valve located at the end of the robotic arm is controlled to execute the colloid valve control commands.

[0019] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the method described in the first aspect.

[0020] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the method described in the first aspect.

[0021] In this embodiment, during the adhesive application process, the robot arm's position data is acquired in real time, and laser detection data is generated by a visual recognition component and a laser detection head located at the end of the robot arm. Based on the laser detection data and the robot arm's position data, a colloid point cloud image in the robot arm coordinate system is generated, and a standard colloid model is generated based on the colloid point cloud image. Based on the colloid point cloud image and the standard colloid model, colloid height loss information is determined, and based on the colloid height loss information, a glue replenishment path trajectory for the robot arm and the glue dispensing speed corresponding to each glue replenishment position on the glue replenishment path trajectory are generated. The above-described glue replenishment information determination method based on laser detection and visual recognition accurately acquires colloid height loss information through laser and vision fusion technology, achieving precise quantitative determination of glue replenishment information and solving the problems of inaccurate positioning and unbalanced quantity control in traditional glue replenishment methods. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating a method for determining adhesive filling information based on laser detection and visual recognition, provided in an embodiment of this application. Figure 2 This is a flowchart illustrating another method for determining adhesive filling information based on laser detection and visual recognition provided in an embodiment of this application. Figure 3 This is a flowchart illustrating another method for determining adhesive filling information based on laser detection and visual recognition provided in this application embodiment; Figure 4 This is an example diagram of a highly missing matrix provided in an embodiment of this application; Figure 5 This is a schematic diagram of a glue filling information determination system based on laser detection and visual recognition provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, specific embodiments of this application will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely for explaining this application and not for limiting it. It should also be noted that, for ease of description, only the parts relevant to this application are shown in the drawings, not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0024] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0025] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0026] The following description, in conjunction with the accompanying drawings, details the method and system for determining glue filling information based on laser detection and visual recognition provided in this application, through specific embodiments and application scenarios.

[0027] First, this application is applicable to high-precision adhesive application scenarios in the industrial manufacturing field, specifically including but not limited to automotive parts assembly, electronic device housing bonding, and aerospace component sealing, where high precision in adhesive forming and thickness consistency are required. It is particularly suitable for adhesive application and touch-up operations on complex curved surfaces and irregular joints. Based on these application scenarios, it is understood that the implementing entity of this application can be an industrial robot control system, a dedicated controller for adhesive application equipment, or an embedded control unit integrating motion control, vision processing, and laser detection modules.

[0028] Figure 1 This is a flowchart illustrating a method for determining adhesive filling information based on laser detection and visual recognition, provided in an embodiment of this application. Figure 1 As shown, the specific steps include the following: S101 acquires the position data of the robotic arm in real time during the adhesive application process, and generates laser detection data by using a vision recognition component and a laser detection head set at the end of the robotic arm.

[0029] The robotic arm can be an electromechanical integrated device that can simulate the movement functions of the human upper limbs and achieve spatial positioning, grasping, handling, and manipulation through preset programs or real-time control signals. Its core features are multi-degree-of-freedom motion capability, precise positioning accuracy, and load capacity adapted to the work scenario. It can independently or collaboratively complete various automated work tasks. The robotic arm adhesive application process can be a process in which the robotic arm moves along a preset path trajectory, and the adhesive flow control valve at its end continuously or intermittently extrudes adhesive into the adhesive application area to form an adhesive layer of a preset shape.

[0030] The position data of the robotic arm can be the angle data of each joint of the robotic arm, the three-dimensional coordinates and attitude data of the end effector (colloidal flow control valve) in the robotic arm coordinate system, or the coordinates of the PLC (programmable logic controller).

[0031] In one embodiment, the position data of the robotic arm can be obtained by collecting the angle data of each joint through the encoder built into the robotic arm, and combining it with the kinematic model of the robotic arm to obtain the real-time three-dimensional coordinates and attitude data of the end effector through forward kinematics calculation.

[0032] Among them, the visual recognition component can be an optoelectronic sensing device that integrates image acquisition, optical processing, signal conversion and data preprocessing functions, and can acquire visual information of target scenes or objects, such as industrial CCD (Charge-Coupled Device) cameras, 3D structured light cameras or binocular vision sensors.

[0033] The end effector of the robotic arm can be a flange of the robotic arm and an end effector mounting base fixed on the flange.

[0034] Among them, the laser detection head can be a high-precision detection device that integrates laser emission, signal reception, and distance measurement functions, and can obtain the three-dimensional contour or distance information of the target surface through the principle of laser reflection.

[0035] The laser detection data can be the three-dimensional spatial coordinate data of key sampling points on the surface of the adhesive within the coating area, obtained based on the principle of laser reflection, in the detection coordinate system. The detection coordinate system can be a right-handed rectangular coordinate system established with the optical center of the visual recognition component as the origin, the X and Y axes parallel to the imaging plane of the visual recognition component (X-axis along the horizontal direction of the imaging plane, Y-axis along the vertical direction of the imaging plane), and the Z-axis along the optical axis of the visual recognition component (perpendicular to the imaging plane outwards).

[0036] In one embodiment, laser detection data is generated by acquiring data through a visual recognition component and a laser detection head located at the end of a robotic arm. This can be achieved by controlling the laser detection head at the end of the robotic arm to emit a laser, and controlling the visual recognition component to acquire two-dimensional image data of the adhesive-coated area in real time. A preset visual recognition algorithm is then used to extract laser point pixel position information from the two-dimensional image data. Based on a pre-built calibration relationship, the three-dimensional relative coordinates of the laser point pixel position information in the detection coordinate system are determined as the laser detection data. The preset visual recognition algorithm may include edge detection algorithms, grayscale transformation algorithms, morphological processing algorithms, and AI (Artificial Intelligence) segmentation algorithms, etc.

[0037] S102, Based on the laser detection data and the robotic arm position data, generate a colloidal point cloud image in the robotic arm coordinate system, and generate a standard colloidal model based on the colloidal point cloud image.

[0038] The robotic arm coordinate system can be a right-handed rectangular coordinate system with the center of the robotic arm base as the origin, the X-axis along the horizontal extension direction of the robotic arm, the Y-axis perpendicular to the X-axis on the horizontal plane, and the Z-axis along the vertical direction.

[0039] The colloidal point cloud image can be a point cloud dataset composed of laser detection data from each frame, with each point cloud data containing the spatial coordinate information of a certain location on the colloidal surface.

[0040] In one embodiment, the method of generating a colloidal point cloud image in the robotic arm coordinate system based on laser detection data and robotic arm position data can be achieved by converting each frame of laser detection data into the robotic arm coordinate system based on pre-stored hand-eye calibration parameters and robotic arm position data, i.e., a colloidal point cloud image in the robotic arm coordinate system.

[0041] Among them, the standard colloidal model can be an idealized three-dimensional colloidal model generated by data fitting and morphological optimization based on the spatial distribution characteristics of effective point cloud data in colloidal point cloud images. This model eliminates defects such as depressions, protrusions, and bubbles on the colloidal surface caused by coating errors and environmental interference, while retaining the overall contour morphological characteristics of the colloidal body.

[0042] In one embodiment, the method of generating a standard colloidal model based on colloidal point cloud images can be achieved by using a radius filtering algorithm to remove isolated noise points in the colloidal point cloud images, using the RANSAC algorithm to fit the overall contour trend of the point cloud data, extracting the two-dimensional path centerline and width data of the colloidal body, and then generating a continuous and smooth colloidal contour path based on the two-dimensional path centerline and width data through B-spline curve interpolation. Finally, the theoretical thickness of the colloidal body is determined based on the height statistics of each point cloud data in the colloidal point cloud image. Combining the smoothed colloidal contour path and the theoretical thickness, an idealized three-dimensional model with a flat surface, regular contour, and uniform thickness is constructed, which is the standard colloidal model.

[0043] S103, based on the colloidal point cloud image and the standard colloidal model, determine the colloidal height missing information, and based on the colloidal height missing information, generate the glue replenishment path trajectory of the robotic arm and the glue dispensing speed corresponding to each glue replenishment position on the glue replenishment path trajectory.

[0044] Among them, the missing height information can be the difference between the actual height of each point cloud data in the colloidal point cloud image and the preset height of the corresponding position in the standard colloidal model.

[0045] In one embodiment, the method for determining the missing height information of colloids based on colloidal point cloud images and standard colloidal models can be as follows: the colloidal point cloud images and standard colloidal models are compared point by point, the height difference between corresponding points is calculated, points with positive height differences are marked as missing points, adjacent missing points are clustered into missing regions using a region growing algorithm, the boundary coordinates, maximum missing depth, average missing depth and missing area of ​​each missing region are determined, and finally, height missing information containing the location, shape and depth distribution of missing regions is generated.

[0046] Among them, the glue-applying path trajectory of the robotic arm can be a glue-applying movement path of the robotic arm end effector planned based on the height missing information.

[0047] The extrusion speed corresponding to each glue-filling position on the glue-filling path trajectory can be the glue extrusion speed calculated based on the height missing depth of each glue-filling position on the glue-filling path trajectory.

[0048] In one embodiment, the method of generating the glue replenishment path trajectory of the robotic arm and the glue dispensing speed corresponding to each glue replenishment position on the glue replenishment path trajectory based on the missing colloid height information can be achieved by using B-spline curve fitting to generate a smooth glue replenishment path trajectory based on the boundary coordinates of the missing area, calculating the required glue dispensing amount at each glue replenishment position based on the missing depth and viscosity parameters of the colloid using fluid dynamics formulas, and then calculating the glue dispensing speed corresponding to each glue replenishment position based on the glue replenishment path movement speed and the required glue dispensing amount.

[0049] Optionally, after generating the glue replenishment path trajectory of the robotic arm based on the glue height loss information and the glue dispensing speed corresponding to each glue replenishment position on the glue replenishment path trajectory, the method further includes: Based on the glue application path trajectory, a robotic arm motion command is generated, and based on the glue dispensing speed corresponding to each glue application position, a glue valve control command is generated. The robotic arm motion commands and the glue valve control commands are time-synchronized and calibrated. The robotic arm is controlled to execute the robotic arm movement commands, and the colloid flow control valve located at the end of the robotic arm is controlled to execute the colloid valve control commands.

[0050] Among them, the robotic arm motion commands can be a standardized set of commands that includes the motion angles, speeds, accelerations, and motion modes of each joint of the robotic arm corresponding to the glue application path trajectory.

[0051] In one embodiment, the method of generating robotic arm motion commands based on the glue application path trajectory can be achieved by calculating the target angle sequence of each joint of the robotic arm based on the three-dimensional coordinate point sequence of the glue application path trajectory through inverse kinematics, and using a trapezoidal velocity planning algorithm to generate the motion velocity curves and acceleration curves of each joint of the robotic arm in combination with preset motion constraint parameters. The target angle, velocity, and acceleration data are then encoded according to the instruction format supported by the robotic arm controller to form complete robotic arm motion commands.

[0052] The glue valve control command can be a control signal that includes the opening and closing timing of the glue flow control valve and the valve opening parameters corresponding to each glue filling position.

[0053] In one embodiment, the method of generating glue valve control commands based on the glue dispensing speed corresponding to each glue replenishment position can be achieved by pre-establishing a calibration mapping table of glue dispensing speed and valve opening parameters, querying the calibration mapping table according to the glue dispensing speed corresponding to each glue replenishment position to obtain the corresponding valve opening parameters, thereby obtaining a valve opening parameter sequence, and encoding the valve opening parameter sequence according to the instruction format supported by the glue flow control valve driver to form a complete glue valve control command.

[0054] In one embodiment, the method for time synchronization calibration of the robotic arm motion commands and the glue valve control commands can be to set a timestamp for the glue valve control commands based on the arrival time of each glue replenishment position in the robotic arm motion commands.

[0055] Among them, the colloidal flow control valve can be an electromagnetic proportional flow valve or a servo control flow valve with high-speed response characteristics.

[0056] In one embodiment, the method of controlling the robotic arm to execute robotic arm motion commands and controlling the colloid flow control valve set at the end of the robotic arm to execute colloid valve control commands can be achieved by sending the robotic arm motion commands to the robotic arm controller in real time via EtherCAT or Profinet industrial bus. The robotic arm controller drives the servo motors of each joint to move according to the commands and provides real-time feedback on the motion status. At the same time, the colloid valve control commands are sent to the driver of the colloid flow control valve via CAN bus, and the driver adjusts the valve opening according to the commands.

[0057] The advantage of this solution is that by accurately generating and calibrating the motion commands of the robotic arm and the control commands of the glue valve, dynamic matching between the movement of the glue replenishment path and the glue dispensing speed is achieved, avoiding problems such as glue replenishment position deviation, insufficient or excessive glue dispensing.

[0058] In this embodiment, during the adhesive application process, the robot arm's position data is acquired in real time, and laser detection data is generated by a visual recognition component and a laser detection head located at the end of the robot arm. Based on the laser detection data and the robot arm's position data, a colloid point cloud image in the robot arm coordinate system is generated, and a standard colloid model is generated based on the colloid point cloud image. Based on the colloid point cloud image and the standard colloid model, colloid height loss information is determined, and based on the colloid height loss information, a glue replenishment path trajectory for the robot arm and the glue dispensing speed corresponding to each glue replenishment position on the glue replenishment path trajectory are generated. The above-described glue replenishment information determination method based on laser detection and visual recognition accurately acquires colloid height loss information through laser and vision fusion technology, achieving precise quantitative determination of glue replenishment information and solving the problems of inaccurate positioning and unbalanced quantity control in traditional glue replenishment methods.

[0059] Figure 2 This is a flowchart illustrating another method for determining adhesive filling information based on laser detection and visual recognition provided in an embodiment of this application. Figure 2 As shown, the specific steps include the following: S201 acquires the position data of the robotic arm in real time during the adhesive application process, and generates laser detection data by using a vision recognition component and a laser detection head set at the end of the robotic arm.

[0060] S202, Generate a colloidal point cloud image in the robotic arm coordinate system based on the laser detection data and the robotic arm position data.

[0061] S203, a clustering algorithm is used to segment the colloidal point cloud image into multiple point cloud clusters, and the three-dimensional centroids of the multiple point cloud clusters are connected to obtain the centerline trajectory.

[0062] Among them, the clustering algorithm can be a density-based spatial clustering algorithm (DBSCAN), a K-Means clustering algorithm, or a Euclidean clustering algorithm. Its core is to group the point cloud data in the colloidal point cloud image according to the spatial distance and density distribution characteristics, so as to achieve effective segmentation of the point cloud.

[0063] Among them, a point cloud cluster can be a set of spatially adjacent point cloud data obtained by clustering algorithm in a colloidal point cloud image. Each point cloud cluster corresponds to the local point cloud data of a continuous area on the colloidal surface, which can reflect the local features of the colloidal body.

[0064] In one embodiment, a clustering algorithm is used to segment a colloidal point cloud image into multiple point cloud clusters. This can be achieved by first setting a clustering radius threshold and a minimum point cloud quantity threshold, then traversing all point cloud data in the colloidal point cloud image, taking any unclassified point as the core, and including adjacent points that are less than the clustering radius threshold in the same candidate cluster. When the number of point cloud data in the candidate cluster reaches the minimum point cloud quantity threshold, a valid point cloud cluster is formed. The above process is repeated until all point cloud data are classified or marked as isolated noise (isolated noise is directly removed), and finally multiple non-overlapping point cloud clusters are obtained.

[0065] The three-dimensional centroid of a point cloud cluster can be the spatial point corresponding to the arithmetic mean of the three-dimensional coordinates of all point cloud data in a single point cloud cluster.

[0066] Among them, the centerline trajectory can be a continuous spatial curve formed by connecting the three-dimensional centroids of all point cloud clusters. Its shape is consistent with the overall extension path of the colloid, which can accurately reflect the core direction of the colloid and provide a reference path for subsequent slicing.

[0067] In one embodiment, the method of connecting the three-dimensional centroids of multiple point cloud clusters to obtain the centerline trajectory can be achieved by first sorting the three-dimensional centroids of all point cloud clusters according to the temporal sequence of the robotic arm's adhesive application or the spatial distribution sequence of the point cloud clusters, and then using a B-spline curve interpolation algorithm to smoothly fit the sorted three-dimensional centroid points to generate a continuous, inflection-free spatial curve, which is the centerline trajectory.

[0068] S204, following the centerline trajectory, the colloidal point cloud image is sliced ​​at equal intervals to obtain point cloud parameter information for multiple cross-sectional point clouds; wherein, the point cloud parameter information includes point cloud density and point cloud radial distance distribution.

[0069] The cross-sectional point cloud can be colloidal point cloud segments taken along the centerline trajectory at preset equal intervals (e.g., 1 mm) perpendicular to the tangent direction of the centerline trajectory. The point cloud parameter information of the cross-sectional point cloud can be quantitative data characterizing the distribution features, density, and contour dimensions of the cross-sectional point cloud, and may include point cloud density and point cloud radial distance distribution. Specifically, point cloud density can be the number of point cloud data points per unit area in the cross-sectional point cloud; the point cloud radial distance distribution can be the statistical distribution data of the radial distances from each point cloud data point in the cross-sectional point cloud to the center of the corresponding cross-section of the centerline trajectory.

[0070] In one embodiment, the method of slicing the colloidal point cloud image at equal intervals along the centerline trajectory to obtain point cloud parameter information for multiple cross-sectional point clouds can be achieved by taking a slice reference point at preset intervals along the extension direction of the centerline trajectory, constructing a virtual slice plane perpendicular to the tangent direction of the centerline trajectory with the slice reference point as the center, filtering out the point cloud data in the colloidal point cloud image that fall within the virtual slice plane to form cross-sectional point clouds, and calculating the point cloud density and radial distance distribution of each cross-sectional point cloud to finally obtain the point cloud parameter information for each cross-sectional point cloud.

[0071] S205, determine standard cross-sectional information based on the point cloud parameter information of the multiple cross-sectional point clouds.

[0072] The standard cross-section information can be an idealized set of cross-section parameters obtained through statistical analysis and optimization based on the point cloud parameter information of all cross-section point clouds.

[0073] In one embodiment, the method for determining standard cross-sectional information based on point cloud parameter information of multiple cross-sectional point clouds can be as follows: first, abnormal cross-sectional point clouds are removed; then, the arithmetic mean of the point cloud density of the remaining effective cross-sectional point clouds is taken as the standard point cloud density; and the median of the average radial distance of the effective cross-sectional point clouds is taken as the standard average radial distance, thus obtaining the standard cross-sectional information.

[0074] Optionally, determining the standard cross-sectional information based on the point cloud parameter information of the plurality of cross-sectional point clouds includes: Cross-sectional point clouds with point cloud density exceeding a preset point cloud density threshold are identified as candidate cross-sectional point clouds. Based on the radial distance distribution of the point cloud of the candidate cross-section, the first principal direction and the second principal direction of the point cloud of the candidate cross-section are determined, and the first radial distance in the first principal direction and the second radial distance in the second principal direction are obtained; The average first principal direction is determined based on the first principal direction, the average second principal direction is determined based on the second principal direction, the average first radial distance is determined based on the first radial distance, and the average second radial distance is determined based on the second radial distance to obtain standard cross-sectional information.

[0075] The preset point cloud density threshold can be used to filter out the point cloud density value of the cross-sectional point cloud that has been sufficiently collected and can truly reflect the characteristics of the colloidal cross section.

[0076] Among them, if the point cloud density of the cross-sectional point cloud exceeds the preset point cloud density threshold, it means that the point cloud data of the cross-sectional point cloud is densely distributed and can fully cover the contour range of the colloidal cross section. The cross-sectional features reflected have high authenticity and reliability. Therefore, the cross-sectional point cloud with a point cloud density exceeding the preset point cloud density threshold can be identified as a candidate cross-sectional point cloud.

[0077] The first principal direction can be the direction in which the radial distance distribution of the point cloud in the candidate cross-section is most dispersed (i.e., the direction of the major axis of the cross-section profile), and the second principal direction can be the direction in which the radial distance distribution of the point cloud is less dispersed (i.e., the direction of the minor axis of the cross-section profile).

[0078] In one embodiment, the method of determining the first principal direction and the second principal direction of the candidate cross-sectional point cloud based on the radial distance distribution of the point cloud can be achieved by calculating the set of radial vectors from all point cloud data in the candidate cross-sectional point cloud to the center of the cross-section, constructing the covariance matrix of the radial vectors, performing eigenvalue decomposition on the covariance matrix to obtain the eigenvectors corresponding to the two largest eigenvalues, determining the eigenvector corresponding to the largest eigenvalue as the first principal direction, and determining the eigenvector corresponding to the second largest eigenvalue as the second principal direction.

[0079] The first radial distance can be the distance from the center of the cross-section to the farthest point cloud data in the first principal direction of the candidate cross-section point cloud, and the second radial distance can be the distance from the center of the cross-section to the farthest point cloud data in the second principal direction of the candidate cross-section point cloud.

[0080] In one embodiment, the method for obtaining the first radial distance in the first principal direction and the second radial distance in the second principal direction can be as follows: construct rays along the first principal direction and the second principal direction respectively, traverse all point cloud data in the candidate cross-section point cloud, calculate the projection distance of each point cloud data in the corresponding principal direction (i.e., the component of the radial distance from the point to the center of the cross-section in the principal direction), take the maximum value of the projection distance in the first principal direction as the first radial distance, and take the maximum value of the projection distance in the second principal direction as the second radial distance.

[0081] The average first principal direction can be a unified direction vector obtained by averaging the first principal directions of all candidate cross-sectional point clouds, and the average second principal direction can be a unified direction vector obtained by averaging the second principal directions of all candidate cross-sectional point clouds.

[0082] In one embodiment, the method of determining the average first principal direction based on the first principal direction and the average second principal direction based on the second principal direction can be achieved by normalizing the first principal direction vector and the second principal direction vector of each candidate cross-sectional point cloud (converting them to unit vectors), then calculating the arithmetic mean of all normalized first principal direction vectors to obtain the average first principal direction vector, calculating the arithmetic mean of all normalized second principal direction vectors to obtain the average second principal direction vector, and finally normalizing the average first principal direction vector and the average second principal direction vector to determine the final average first principal direction and average second principal direction.

[0083] The average first radial distance can be the value obtained by statistically averaging the first radial distances of all candidate cross-sectional point clouds, and the average second radial distance can be the value obtained by statistically averaging the second radial distances of all candidate cross-sectional point clouds.

[0084] In one embodiment, the method of determining the average first radial distance based on the first radial distance and the average second radial distance based on the second radial distance can be achieved by removing outliers of the first radial distance and the second radial distance in the candidate cross-sectional point cloud, then calculating the arithmetic mean of the remaining valid first radial distances to obtain the average first radial distance, and calculating the arithmetic mean of the remaining valid second radial distances to obtain the average second radial distance.

[0085] The advantage of this design is that it is compatible with various cross-sectional types of colloids, such as circles and ellipses, significantly improving the accuracy and adaptability of standard cross-sectional information and laying a reliable foundation for the subsequent generation of standard colloid models.

[0086] S206, Generate a standard colloidal model based on the standard cross-sectional information and the centerline trajectory.

[0087] In one embodiment, the method of generating a standard colloidal model based on standard cross-sectional information and centerline trajectory can be achieved by generating a continuous cylindrical structure (wherein the surface subdivision factor of the cylindrical structure is equal to the standard point cloud density) along the extension direction of the centerline trajectory according to the standard average radial distance in the standard cross-sectional information, i.e., a standard colloidal model.

[0088] S207, Based on the colloidal point cloud image and the standard colloidal model, determine the colloidal height missing information, and based on the colloidal height missing information, generate the glue replenishment path trajectory of the robotic arm and the glue dispensing speed corresponding to each glue replenishment position on the glue replenishment path trajectory.

[0089] The advantage of this approach is that it can accurately capture the actual path and cross-sectional features of the colloid. The standard colloid model generated based on these features is closer to the actual shape of the colloid, avoiding the problem of excessive deviation between the preset model and the actual coating trajectory.

[0090] Figure 3 This is a flowchart illustrating another method for determining adhesive filling information based on laser detection and visual recognition provided in this application. Figure 3 As shown, the specific steps include the following: S301 acquires the position data of the robotic arm in real time during the adhesive application process and generates laser detection data by using a vision recognition component and a laser detection head set at the end of the robotic arm.

[0091] S302, Based on the laser detection data and the robotic arm position data, generate a colloidal point cloud image in the robotic arm coordinate system, and generate a standard colloidal model based on the colloidal point cloud image.

[0092] S303, Align the standard colloidal model to the colloidal point cloud image to obtain a standard point cloud image, compare the colloidal point cloud image and the standard point cloud image to obtain a height missing matrix, and calculate the height missing gradient matrix corresponding to the height missing matrix.

[0093] The standard point cloud image can be the set of three-dimensional coordinates of a theoretically perfect colloidal surface in the same coordinate system, obtained by spatially registering an ideal standard colloidal model with an actual colloidal point cloud image.

[0094] In one embodiment, the standard colloidal model is aligned to the colloidal point cloud image to obtain the standard point cloud image. This can be achieved by using an iterative nearest-point algorithm, which minimizes the overall distance between the surface of the standard colloidal model and the colloidal point cloud image to complete the spatial registration of the two 3D models.

[0095] The height missing matrix can be a two-dimensional data grid, where each grid point stores the absolute value of the height difference between the colloidal point cloud image and the standard point cloud image in the normal direction.

[0096] In one embodiment, the method of obtaining the height missing matrix by comparing the colloidal point cloud image and the standard point cloud image can be as follows: for each point in the colloidal point cloud image, find the corresponding nearest point in the standard point cloud image, calculate the distance between the two in the normal vector direction, and map all distance values ​​onto a two-dimensional plane to form a height missing matrix.

[0097] Figure 4 This is an example diagram of a highly missing matrix provided in an embodiment of this application. For example... Figure 4 As shown, the height loss matrix is ​​presented in grayscale form. The darker the area, the more severe the height loss of the colloid, and the lighter the area, the more the colloid accumulation meets or exceeds the standard.

[0098] The height missing gradient matrix can be a matrix obtained by calculating the spatial gradient of the height missing matrix, and is used to characterize the height change rate and direction of each point in the height missing matrix in its neighborhood.

[0099] In one embodiment, the height missing gradient matrix corresponding to the height missing matrix can be calculated by performing a two-dimensional convolution operation between the image gradient operator such as Sobel or Prewitt and the height missing matrix to obtain the gradient components in the X and Y directions, and then calculating the total gradient magnitude to obtain the height missing gradient matrix.

[0100] S304, Based on the height missing gradient matrix, identify and filter out the high gradient noise regions in the height missing matrix to obtain the target height missing matrix.

[0101] Among them, the high gradient noise region in the height missing matrix can be caused by factors such as point cloud acquisition noise, colloidal surface reflection or attached bubbles, and appears as an isolated, abrupt region with an abnormally steep boundary in the height missing matrix.

[0102] The target height missing matrix can be a clean height missing matrix that contains only real, continuous colloidal missing regions after filtering out high gradient noise regions.

[0103] In one embodiment, the method of identifying and filtering high-gradient noise regions in the high-deficient gradient matrix based on the high-deficient gradient matrix to obtain the target high-deficient matrix can be achieved by setting a gradient magnitude threshold, marking regions in the high-deficient gradient matrix whose gradient magnitude exceeds the threshold as masks, and using the masks to filter the original high-deficient matrix, setting the values ​​of these regions to zero or filling them with the neighborhood mean to obtain the target high-deficient matrix.

[0104] S305, Based on the information about the missing colloid height, generate the colloid replenishment path trajectory of the robotic arm and the colloid dispensing speed corresponding to each colloid replenishment position on the colloid replenishment path trajectory.

[0105] Optionally, generating the glue replenishment path trajectory of the robotic arm and the glue dispensing speed corresponding to each glue replenishment position on the glue replenishment path trajectory based on the glue height loss information includes: The set of coordinates of connected elements with values ​​greater than zero in the target height missing matrix is ​​determined as the region to be glued, and the centerline trajectory of the region to be glued is determined as the glued path trajectory. Based on the target height missing matrix, calculate the cross-sectional glue filling amount centered on each glue filling position on the glue filling path trajectory. For each glue filling position, calculate the average cross-sectional glue filling amount of the glue filling position and two adjacent glue filling positions as the target glue output amount corresponding to the glue filling position. Obtain a preset end-effector movement speed, and calculate the glue dispensing speed corresponding to each glue dispensing position on the glue dispensing path trajectory based on the preset end-effector movement speed and the target glue dispensing amount.

[0106] In the target height missing matrix, an element value greater than zero indicates that the colloid height at that coordinate point is lower than the standard height, indicating a height missing.

[0107] The set of connected element coordinates can be a region consisting of spatially adjacent pixels or data points whose element values ​​are all greater than zero, identified by region growing or morphological operations; the region to be glued can be one or more continuous spatial regions that need to be glued, as determined by connectivity analysis.

[0108] In one embodiment, the method of determining the centerline trajectory of the area to be glued as the glue application path trajectory can be achieved by using a skeletonization algorithm or a central axis transformation algorithm to extract the center skeleton line of the area to be glued as the centerline trajectory.

[0109] The amount of glue applied to the cross-section centered on each glue application position on the glue application path trajectory can be the total missing amount obtained by integrating or summing the element values ​​at the corresponding positions in the target height missing matrix on the cross-section perpendicular to the glue application path trajectory.

[0110] In one embodiment, the method of calculating the amount of adhesive applied to the cross section centered on each adhesive application position on the adhesive application path trajectory based on the target height missing matrix can be achieved by accumulating the missing values ​​of all corresponding points in the matrix within a cross section of a preset width centered on each adhesive application position on the adhesive application path trajectory.

[0111] The target glue output at the glue replenishment location can be the planned glue volume allocated to compensate for the height loss at that location and its adjacent areas.

[0112] The adjacent glue replenishment locations can be the upstream glue replenishment location and the downstream glue replenishment location that are closest to the current glue replenishment location on the glue replenishment path trajectory.

[0113] In one embodiment, for each glue application location, the average of the glue application amount of the glue application location and the cross-sectional glue application amount of two adjacent glue application locations is calculated as the target glue application amount corresponding to the glue application location. This can be achieved by summing the glue application amount of the glue application location and the cross-sectional glue application amount of two adjacent glue application locations, and then dividing the summation result by 3 to obtain the target glue application amount corresponding to the glue application location.

[0114] The preset end-effector movement speed can be a constant movement speed of the end of the robotic arm when performing glue application, which is preset according to process requirements.

[0115] In one embodiment, the preset end-effector movement speed can be obtained by directly reading a pre-built database of adhesive application process parameters.

[0116] In one embodiment, the glue dispensing speed corresponding to each glue dispensing position on the glue dispensing path trajectory can be calculated based on the preset end-effector movement speed and the target glue dispensing amount. This can be achieved by dividing the target glue dispensing amount corresponding to each glue dispensing position on the glue dispensing path trajectory by the preset end-effector movement speed.

[0117] Optionally, the step of calculating the dispensing speed corresponding to each dispensing position on the dispensing path trajectory based on the preset robotic arm end-effector movement speed and the target dispensing amount includes: Based on the preset end-effector movement speed and the target glue dispensing amount, the basic glue dispensing speed corresponding to each glue dispensing position on the glue dispensing path trajectory is determined. Obtain colloidal formulation information, and determine the colloidal viscosity compensation coefficient and the colloidal curing compensation coefficient based on the colloidal formulation information; The dispensing speed at each dispensing position on the dispensing path trajectory is calculated based on the colloid viscosity compensation coefficient, the colloid curing compensation coefficient, and the basic dispensing speed.

[0118] The basic dispensing speed can be the theoretical dispensing speed calculated based on the end effector speed of the robotic arm to meet the target dispensing volume requirement, without considering the influence of the adhesive's own properties.

[0119] In one embodiment, the method of determining the basic dispensing speed corresponding to each dispensing position on the dispensing path trajectory based on the preset end-effector movement speed and the target dispensing amount can be achieved by dividing the target dispensing amount corresponding to each dispensing position on the dispensing path trajectory by the preset end-effector movement speed to obtain the basic dispensing speed corresponding to each dispensing position on the dispensing path trajectory.

[0120] The colloidal formulation information can be a set of parameters used to identify the type of colloidal material, the proportion of components, and key physicochemical properties, such as colloidal type, viscosity grade, and curing rate.

[0121] In one embodiment, the colloidal formulation information can be obtained by directly reading a pre-built database of coating process parameters.

[0122] The colloid viscosity compensation coefficient can be used to correct the dispensing speed to compensate for the effect of flow resistance caused by colloid viscosity; the colloid curing compensation coefficient can be used to correct the dispensing speed to compensate for the effect of colloid curing characteristics (such as thixotropy and curing shrinkage).

[0123] In one embodiment, the method for determining the colloidal viscosity compensation coefficient and the colloidal curing compensation coefficient based on the colloidal formulation information can be achieved by querying a pre-built mapping table of colloidal formulation information, colloidal viscosity compensation coefficient, and colloidal curing compensation coefficient, and directly obtaining the corresponding colloidal viscosity compensation coefficient and colloidal curing compensation coefficient based on the current colloidal formulation information.

[0124] In one embodiment, the dispensing speed corresponding to each dispensing position on the dispensing path trajectory can be calculated based on the colloid viscosity compensation coefficient, the colloid curing compensation coefficient, and the basic dispensing speed. This can be achieved by multiplying the colloid viscosity compensation coefficient, the colloid curing compensation coefficient, and the basic dispensing speed to obtain the dispensing speed corresponding to each dispensing position on the dispensing path trajectory.

[0125] The advantage of this scheme is that by introducing the characteristics of the colloidal formulation to dynamically compensate for the dispensing process, the accuracy and adaptability of dispensing control can be significantly improved, effectively overcoming process fluctuations caused by changes in colloidal viscosity and curing behavior.

[0126] The advantage of this solution is that it enables precise and adaptive automated glue replenishment by dynamically adjusting the glue dispensing speed.

[0127] The advantage of this solution is that by introducing gradient analysis to intelligently filter out noise, it can accurately identify the actual areas of missing adhesive. Based on this, the generated adhesive replenishment path trajectory and dispensing speed can further help to achieve quantitative and adaptive compensation for the missing adhesive areas, avoiding mis-replenishment or missed replenishment caused by noise interference, and effectively ensuring the adhesive coating quality of the product.

[0128] Figure 5 This is a schematic diagram of a glue filling information determination system based on laser detection and visual recognition, provided in an embodiment of this application. Figure 5 As shown, the system includes: The data acquisition module 510 is used to acquire the position data of the robotic arm in real time during the adhesive application process, and to collect and generate laser detection data through the vision recognition component and the laser detection head set at the end of the robotic arm. The model generation module 520 is used to generate a colloidal point cloud image in the robotic arm coordinate system based on the laser detection data and the robotic arm position data, and to generate a standard colloidal model based on the colloidal point cloud image. The information determination module 530 is used to determine the missing colloid height information based on the colloid point cloud image and the standard colloid model, and to generate the colloid replenishment path trajectory of the robotic arm and the dispensing speed corresponding to each colloid replenishment position on the colloid replenishment path trajectory based on the missing colloid height information.

[0129] Optionally, the model generation module 520 is specifically used for: The colloidal point cloud image is segmented into multiple point cloud clusters using a clustering algorithm, and the three-dimensional centroids of the multiple point cloud clusters are connected to obtain the centerline trajectory. Following the centerline trajectory, the colloidal point cloud image is sliced ​​at equal intervals to obtain point cloud parameter information for multiple cross-sectional point clouds; wherein, the point cloud parameter information includes point cloud density and point cloud radial distance distribution; Standard cross-sectional information is determined based on the point cloud parameter information of the multiple cross-sectional point clouds; A standard colloidal model is generated based on the standard cross-sectional information and the centerline trajectory.

[0130] Optionally, the model generation module 520 is specifically used for: Cross-sectional point clouds with point cloud density exceeding a preset point cloud density threshold are identified as candidate cross-sectional point clouds. Based on the radial distance distribution of the point cloud of the candidate cross-section, the first principal direction and the second principal direction of the point cloud of the candidate cross-section are determined, and the first radial distance in the first principal direction and the second radial distance in the second principal direction are obtained; The average first principal direction is determined based on the first principal direction, the average second principal direction is determined based on the second principal direction, the average first radial distance is determined based on the first radial distance, and the average second radial distance is determined based on the second radial distance to obtain standard cross-sectional information.

[0131] Optionally, the information determination module 530 is specifically used for: Align the standard colloidal model to the colloidal point cloud image to obtain a standard point cloud image. Compare the colloidal point cloud image and the standard point cloud image to obtain a height missing matrix, and calculate the height missing gradient matrix corresponding to the height missing matrix. Based on the height missing gradient matrix, high gradient noise regions in the height missing matrix are identified and filtered out to obtain the target height missing matrix.

[0132] Optionally, the information determination module 530 is specifically used for: The set of coordinates of connected elements with values ​​greater than zero in the target height missing matrix is ​​determined as the region to be glued, and the centerline trajectory of the region to be glued is determined as the glued path trajectory. Based on the target height missing matrix, calculate the cross-sectional glue filling amount centered on each glue filling position on the glue filling path trajectory. For each glue filling position, calculate the average cross-sectional glue filling amount of the glue filling position and two adjacent glue filling positions as the target glue output amount corresponding to the glue filling position. Obtain a preset end-effector movement speed, and calculate the glue dispensing speed corresponding to each glue dispensing position on the glue dispensing path trajectory based on the preset end-effector movement speed and the target glue dispensing amount.

[0133] Optionally, the information determination module 530 is specifically used for: Based on the preset end-effector movement speed and the target glue dispensing amount, the basic glue dispensing speed corresponding to each glue dispensing position on the glue dispensing path trajectory is determined. Obtain colloidal formulation information, and determine the colloidal viscosity compensation coefficient and the colloidal curing compensation coefficient based on the colloidal formulation information; The dispensing speed at each dispensing position on the dispensing path trajectory is calculated based on the colloid viscosity compensation coefficient, the colloid curing compensation coefficient, and the basic dispensing speed.

[0134] Optionally, the device is also used for: Based on the glue application path trajectory, a robotic arm motion command is generated, and based on the glue dispensing speed corresponding to each glue application position, a glue valve control command is generated. The robotic arm motion commands and the glue valve control commands are time-synchronized and calibrated. The robotic arm is controlled to execute the robotic arm movement commands, and the colloid flow control valve located at the end of the robotic arm is controlled to execute the colloid valve control commands.

[0135] In this embodiment, a data acquisition module is used to acquire real-time position data of the robotic arm during the adhesive application process, and generate laser detection data through a visual recognition component and a laser detection head set at the end of the robotic arm. A model generation module is used to generate a colloid point cloud image in the robotic arm coordinate system based on the laser detection data and the robotic arm position data, and to generate a standard colloid model based on the colloid point cloud image. An information determination module is used to determine colloid height loss information based on the colloid point cloud image and the standard colloid model, and to generate the adhesive replenishment path trajectory of the robotic arm and the dispensing speed corresponding to each adhesive replenishment position on the adhesive replenishment path trajectory based on the colloid height loss information. The aforementioned adhesive replenishment information determination system based on laser detection and visual recognition accurately acquires colloid height loss information through laser and vision fusion technology, achieving precise quantitative determination of adhesive replenishment information and solving the problems of inaccurate positioning and unbalanced quantity control in traditional adhesive replenishment methods.

[0136] The glue filling information determination system based on laser detection and visual recognition in this application embodiment can be a system, or a component, integrated circuit, or chip in a terminal. This system can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.

[0137] The glue filling information determination system based on laser detection and visual recognition in this application embodiment can be a system with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0138] The glue filling information determination system based on laser detection and visual recognition provided in this application embodiment can realize the various processes implemented in the above embodiments. To avoid repetition, it will not be described again here.

[0139] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. For example... Figure 6As shown, this application embodiment also provides an electronic device 600, including a processor 601, a memory 602, and a program or instructions stored in the memory 602 and executable on the processor 601. When the program or instructions are executed by the processor 601, they implement the various processes of the above-described embodiment of the method for determining glue filling information based on laser detection and visual recognition, and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0140] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0141] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described embodiment of the method for determining glue filling information based on laser detection and visual recognition, and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0142] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0143] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element. Furthermore, it should be noted that the scope of the methods and systems in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0144] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0145] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0146] The above description is merely a preferred embodiment and the technical principles employed in this application. This application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions that can be made by those skilled in the art will not depart from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of this application, the scope of which is determined by the scope of the claims.

Claims

1. A method for determining adhesive replenishment information based on laser detection and visual recognition, characterized in that, The method includes: The robot arm position data is acquired in real time during the glue application process, and laser detection data is generated by a vision recognition component and a laser detection head set at the end of the robot arm. Based on the laser detection data and the robotic arm position data, a colloidal point cloud image in the robotic arm coordinate system is generated, and a standard colloidal model is generated based on the colloidal point cloud image. Based on the colloidal point cloud image and the standard colloidal model, the missing colloidal height information is determined, and based on the missing colloidal height information, the adhesive replenishment path trajectory of the robotic arm and the dispensing speed corresponding to each adhesive replenishment position on the adhesive replenishment path trajectory are generated.

2. The method for determining glue replenishment information based on laser detection and visual recognition according to claim 1, characterized in that, The generation of a standard colloidal model based on the colloidal point cloud image includes: The colloidal point cloud image is segmented into multiple point cloud clusters using a clustering algorithm, and the three-dimensional centroids of the multiple point cloud clusters are connected to obtain the centerline trajectory. Following the centerline trajectory, the colloidal point cloud image is sliced ​​at equal intervals to obtain point cloud parameter information for multiple cross-sectional point clouds; wherein, the point cloud parameter information includes point cloud density and point cloud radial distance distribution; Standard cross-sectional information is determined based on the point cloud parameter information of the multiple cross-sectional point clouds; A standard colloidal model is generated based on the standard cross-sectional information and the centerline trajectory.

3. The method for determining glue replenishment information based on laser detection and visual recognition according to claim 2, characterized in that, The determination of standard cross-sectional information based on the point cloud parameter information of the multiple cross-sectional point clouds includes: Cross-sectional point clouds with point cloud density exceeding a preset point cloud density threshold are identified as candidate cross-sectional point clouds. Based on the radial distance distribution of the point cloud of the candidate cross-section, the first principal direction and the second principal direction of the point cloud of the candidate cross-section are determined, and the first radial distance in the first principal direction and the second radial distance in the second principal direction are obtained; The average first principal direction is determined based on the first principal direction, the average second principal direction is determined based on the second principal direction, the average first radial distance is determined based on the first radial distance, and the average second radial distance is determined based on the second radial distance to obtain standard cross-sectional information.

4. The method for determining adhesive replenishment information based on laser detection and visual recognition according to any one of claims 1-3, characterized in that, The determination of colloidal height missing information based on the colloidal point cloud image and the standard colloidal model includes: Align the standard colloidal model to the colloidal point cloud image to obtain a standard point cloud image. Compare the colloidal point cloud image and the standard point cloud image to obtain a height missing matrix, and calculate the height missing gradient matrix corresponding to the height missing matrix. Based on the height missing gradient matrix, high gradient noise regions in the height missing matrix are identified and filtered out to obtain the target height missing matrix.

5. The method for determining glue replenishment information based on laser detection and visual recognition according to claim 4, characterized in that, The process of generating the glue replenishment path trajectory for the robotic arm based on the glue height loss information and the glue dispensing speed corresponding to each glue replenishment position on the glue replenishment path trajectory includes: The set of coordinates of connected elements with values ​​greater than zero in the target height missing matrix is ​​determined as the region to be glued, and the centerline trajectory of the region to be glued is determined as the glued path trajectory. Based on the target height missing matrix, calculate the cross-sectional glue filling amount centered on each glue filling position on the glue filling path trajectory. For each glue filling position, calculate the average cross-sectional glue filling amount of the glue filling position and two adjacent glue filling positions as the target glue output amount corresponding to the glue filling position. Obtain a preset end-effector movement speed, and calculate the glue dispensing speed corresponding to each glue dispensing position on the glue dispensing path trajectory based on the preset end-effector movement speed and the target glue dispensing amount.

6. The method for determining glue replenishment information based on laser detection and visual recognition according to claim 5, characterized in that, The step of calculating the dispensing speed corresponding to each dispensing position on the dispensing path trajectory based on the preset robotic arm end-effector movement speed and the target dispensing amount includes: Based on the preset end-effector movement speed and the target glue dispensing amount, the basic glue dispensing speed corresponding to each glue dispensing position on the glue dispensing path trajectory is determined. Obtain colloidal formulation information, and determine the colloidal viscosity compensation coefficient and the colloidal curing compensation coefficient based on the colloidal formulation information; The dispensing speed at each dispensing position on the dispensing path trajectory is calculated based on the colloid viscosity compensation coefficient, the colloid curing compensation coefficient, and the basic dispensing speed.

7. The method for determining glue replenishment information based on laser detection and visual recognition according to claim 1, characterized in that, After generating the glue replenishment path trajectory of the robotic arm based on the glue height loss information and the glue dispensing speed corresponding to each glue replenishment position on the glue replenishment path trajectory, the method further includes: Based on the glue application path trajectory, a robotic arm motion command is generated, and based on the glue dispensing speed corresponding to each glue application position, a glue valve control command is generated. The robotic arm motion commands and the glue valve control commands are time-synchronized and calibrated. The robotic arm is controlled to execute the robotic arm movement commands, and the colloid flow control valve located at the end of the robotic arm is controlled to execute the colloid valve control commands.

8. A system for determining glue application information based on laser detection and visual recognition, characterized in that, The system includes: The data acquisition module is used to acquire the position data of the robotic arm in real time during the adhesive application process, and to collect and generate laser detection data through the vision recognition component and the laser detection head set at the end of the robotic arm. The model generation module is used to generate a colloidal point cloud image in the robotic arm coordinate system based on the laser detection data and the robotic arm position data, and to generate a standard colloidal model based on the colloidal point cloud image. The information determination module is used to determine the missing colloid height information based on the colloid point cloud image and the standard colloid model, and to generate the colloid replenishment path trajectory of the robotic arm and the dispensing speed corresponding to each colloid replenishment position on the colloid replenishment path trajectory based on the missing colloid height information.

9. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the glue filling information determination method based on laser detection and visual recognition as described in any one of claims 1-7.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions, which, when executed by a processor, implement the glue filling information determination method based on laser detection and visual recognition as described in any one of claims 1-7.

Images