A method and system for three-dimensional fusion detection of dynamic phase shift in robot adhesive application
By using dynamic phase analysis and closed-loop calibration with multiple cameras and phase-shifting fringe projection units, the error problem in obtaining three-dimensional parameters in robot adhesive coating inspection was solved, achieving high-precision adhesive path inspection and defect identification.
Patent Information
- Application Number
- CN202611262160.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-19
- Publication Date
- 2026-11-17
AI Technical Summary
Existing robotic adhesive application detection solutions struggle to accurately obtain the three-dimensional geometric parameters of the adhesive path under continuous motion conditions, and the lack of closed-loop calibration between two-dimensional and three-dimensional information leads to detection errors and misjudgments.
Using multiple cameras and phase-shifting fringe projection units, the phase is dynamically analyzed and the two-dimensional texture and three-dimensional data are fused and detected by combining the robot pose and historical 3D data through spatial calibration and image registration. The three-dimensional region is corrected by using the two-dimensional boundary and a local substrate reference surface is established for closed-loop calibration.
It achieves high-precision glue path detection under continuous robot motion conditions, reduces boundary errors, improves the accuracy of glue path width, height and volume calculation, and reduces misjudgment of defects such as glue leakage and glue breakage.
Smart Images

Figure CN123111139A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of automotive manufacturing and visual inspection technology, specifically relating to a three-dimensional fusion detection method and system for dynamic phase shift of a robot applying adhesive. Background Technology
[0002] In automotive body welding, component assembly, and battery tray sealing processes, automated robotic adhesive application is widely used for structural bonding, edge joining, sealing, and corrosion protection. The continuity, position, width, height, and amount of adhesive per unit length of the adhesive path directly affect the subsequent welding, assembly, and sealing effects. Therefore, it is necessary to promptly detect defects such as adhesive leakage, adhesive breaks, adhesive path deviation, excessive adhesive application, insufficient adhesive application, and uneven adhesive application during or after robotic adhesive application.
[0003] In existing adhesive path visual inspection solutions, ordinary two-dimensional machine vision typically determines the adhesive path outline based on the color, grayscale, or edge differences between the adhesive and the substrate. It can detect the position and width of the adhesive path to a certain extent, but it is difficult to directly obtain three-dimensional geometric parameters such as the height, cross-sectional area, and volume of the adhesive path. When the adhesive is transparent, translucent, or close to the color of the substrate, the two-dimensional boundary is also easily affected by reflection and changes in ambient brightness.
[0004] To obtain three-dimensional information about the adhesive path, laser profiling, structured light, or phase-shifting fringe measurement can be used. Phase-shifting fringe measurement typically involves continuously projecting multiple periodic fringes with a predetermined phase difference onto the surface being measured. The wrapping phase is calculated based on the intensity changes of the same spatial point under different phase shift states, and the three-dimensional position is then obtained through absolute phase recovery and spatial calibration. This method has high spatial resolution in static measurements.
[0005] However, in robotic online adhesive application inspection scenarios, the inspection camera and stripe projection device are typically mounted at the robot's end effector and move continuously with the adhesive gun. Within one phase shift cycle, the robot undergoes spatial displacement, causing the pixel position of the same workpiece spatial point to change in different phase shift images. Simultaneously, the stripe projection device itself also moves with the robot, altering the geometric position of the same fixed spatial point relative to the projection device. The actual stripe phase change experienced includes not only the phase shift actively applied by the projection controller but also an additional phase caused by changes in the projection geometry. If the phase is directly analyzed according to the static, equally spaced phase shift relationship, motion phase errors are easily introduced.
[0006] Furthermore, the phase-shifting algorithm directly obtains the wrapping phase. For continuous periodic stripes, different phase orders may correspond to different three-dimensional positions. Under continuous robot motion conditions, if only a single viewpoint is relied upon or only the phase order of the previous detection cycle is used, local reflections, occlusions, or historical errors may cause misjudgment of the phase order and cause the error to propagate along the continuous detection cycle.
[0007] On the other hand, the interface between the colloid and the metal substrate is not only a critical location for 2D contour detection, but also the boundary reference for calculating the adhesive path width, cross-sectional area, and volume. Highly reflective metal surfaces may cause anomalies in 3D phase data at the edges, while the edges of 2D textures may also shift due to transparent adhesive, reflection, or similar colors. In existing solutions, 2D and 3D information are often used for parallel judgment, lacking a closed-loop calibration process that uses 2D boundaries to constrain the 3D region, then uses 3D height to reverse correct the 2D boundaries, and finally redetermines the substrate reference surface.
[0008] Therefore, an online adhesive coating detection solution suitable for continuous robot motion is needed to improve the above-mentioned problems. Summary of the Invention
[0009] In view of the above-mentioned problems in the prior art, the purpose of this invention is to provide a three-dimensional fusion detection method for dynamic phase shift of robot adhesive application.
[0010] A three-dimensional fusion detection method for dynamic phase shift during robot adhesive application includes the following steps: S1. Multiple cameras with overlapping detection fields of view and a phase-shift fringe projection unit are set at the end of the robot to establish a spatial calibration relationship between the multiple cameras, the phase-shift fringe projection unit and the end of the robot; during the movement of the robot along the adhesive application trajectory, the phase-shift fringe projection unit sequentially projects multiple phase-shift fringes with different set phase shift amounts, and the multiple cameras synchronously acquire the corresponding phase-shift images and obtain the robot end pose corresponding to the exposure time of each phase-shift image; S2. Based on the robot end-effector pose at each phase-shift image exposure time and the predicted three-dimensional surface determined by at least one of the pre-stored workpiece three-dimensional reference information and the three-dimensional adhesive path data from the previous detection cycle, multiple phase-shift images within the same phase-shift cycle are registered to the reference exposure time; based on the spatial pose of the phase-shift stripe projection unit at each exposure time, the projection geometric phase change of the same spatial position relative to the reference exposure time is determined, and the projection geometric phase change is combined with the set phase shift amount of the corresponding phase-shift stripe to obtain the effective phase shift amount; S3. Perform dynamic phase analysis on the registered multiple phase-shifted images according to the effective phase shift amount to obtain the wrapping phase under the reference exposure state and the common gray-scale component that is independent of the phase shift state, and form a two-dimensional texture image of the adhesive path detection area according to the common gray-scale component. S4. Establish phase order prediction constraints for the wrapping phase based on at least one of the pre-stored workpiece 3D reference information and historical 3D adhesive path data, and generate candidate phase orders accordingly; recover candidate 3D points based on each candidate phase order, project the candidate 3D points onto at least one other camera, and determine the target phase order based on the periodic phase residual between the predicted wrapping phase corresponding to the candidate 3D point and the measured wrapping phase at the corresponding projection position of the other camera, so as to obtain 3D adhesive path data of the adhesive path detection area. S5. Obtain the initial adhesive path boundary based on the two-dimensional texture image. Correct the adhesive path boundary in the neighborhood of the initial adhesive path boundary by combining the two-dimensional texture gradient and the three-dimensional height gradient of the three-dimensional adhesive path data. Redetermine the adhesive path region and the substrate region based on the corrected adhesive path boundary. Construct a local substrate reference surface using the three-dimensional data of the substrate region. Determine the three-dimensional cross-sectional parameters of the adhesive path based on the local substrate reference surface. S6. Determine the glue path position and glue path geometric parameters based on the robot's glue application trajectory, the corrected glue path boundary, and the glue path three-dimensional cross-sectional parameters, and determine the glue path defect information based on the difference between the glue path position and glue path geometric parameters and the preset glue path parameters.
[0011] Preferably, the plurality of cameras includes three global shutter area array cameras, which are arranged circumferentially at intervals along the detection component at the end of the robot and facing the glue path detection area located behind the direction of movement of the glue gun. The internal parameters and lens distortion parameters of each global shutter array camera, the pose relationship of each global shutter array camera relative to the robot end effector, and the pose relationship of the phase shift fringe projection unit relative to the robot end effector are pre-calibrated. The robot end-effector pose data output by the robot controller is correlated with the phase-shifted image according to a unified time reference. When the sampling time of the robot end-effector pose data is inconsistent with the exposure time of the phase-shifted image, pose interpolation is performed based on the robot end-effector pose at adjacent sampling times.
[0012] Preferably, in step S2: The same spatial point on the predicted three-dimensional surface is projected onto each phase-shift image according to the camera spatial pose corresponding to the exposure time of each phase-shift image. The pixel displacement of the spatial point relative to the reference exposure time is determined, and the phase-shift image is remapped according to the pixel displacement. The spatial points are transformed to the corresponding projection coordinate system according to the spatial pose of the phase-shifting stripe projection unit at each exposure time. The projection base phase is determined according to its projection coordinates, and the projection geometric phase change is obtained based on the projection base phase at the reference exposure time. When the spatial deviation between the three-dimensional adhesive path data obtained in the current detection cycle and the predicted three-dimensional surface exceeds the preset reconstruction deviation threshold, the predicted three-dimensional surface is updated with the current three-dimensional adhesive path data, and the pixel displacement and effective phase shift are re-determined.
[0013] Preferably, in step S3: For the nth registered phase-shifted image corresponding to the same fringe direction, the gray value of the same reference pixel satisfy: ; Where A is a common grayscale component that is independent of the phase shift state. Let be the effective phase shift amount corresponding to the nth phase-shifted image, and C and S be parameters to be determined related to the wrapping phase and fringe modulation amplitude under the reference exposure state. Solve for A, C, and S based on at least three phase shift images with different effective phase shifts, and determine the wrap-around phase. : Stripe tone system M: ; The basic phase quality is determined based on the fringe modulation and phase fitting residual, and the basic phase quality is corrected based on the pixel saturation state.
[0014] Preferably, in step S4: In the initial detection cycle, a candidate three-dimensional spatial range is determined based on the pre-stored three-dimensional reference information of the workpiece and the allowable height range of the adhesive path, and a candidate phase level is determined based on the candidate three-dimensional spatial range; In subsequent detection cycles, the reliable three-dimensional adhesive path data obtained in the previous detection cycle is stored in the robot base coordinate system or the workpiece coordinate system. The predicted absolute phase corresponding to the reliable three-dimensional adhesive path data is determined according to the spatial pose of the phase shift fringe projection unit at the current reference exposure time. The predicted phase level is determined according to the predicted absolute phase and the current wrapping phase. The predicted phase level and its adjacent phase levels form a candidate level set. For each candidate phase level, candidate 3D points are recovered. The predicted wrapping phase is determined based on the spatial relationship between the candidate 3D points and the phase shift fringe projection unit at the reference exposure time. The candidate 3D points are then projected onto other cameras to read the measured wrapping phase at the corresponding positions. The target phase level is determined based on the periodic phase residual between the predicted wrapping phase and the measured wrapping phase.
[0015] Preferably, in step S4: For 3D adhesive path data obtained from different cameras, the basic reliability is determined based on the stripe modulation and multi-view periodic phase consistency, and the basic reliability is corrected based on one or more of the following: pixel saturation state, phase fitting residual, deviation between 3D position and historical 3D prediction data, and phase continuity between adjacent spatial points. When the spatial deviation between 3D data obtained from different cameras in the same spatial area does not exceed a preset consistency threshold, the 3D data are weighted and fused according to their corresponding reliability; when the spatial deviation exceeds the preset consistency threshold, the 3D data with higher reliability is retained.
[0016] Preferably, in step S5: An edge search region is formed by extending along the initial adhesive path boundary to the inner and outer sides of the adhesive path. Multiple candidate boundary positions are determined within the edge search region along the local normal direction of the initial adhesive path boundary. The corrected boundary position is determined based on the two-dimensional texture gradient, three-dimensional height gradient, three-dimensional data reliability, and boundary contour continuity of each candidate boundary position, and the corrected adhesive road boundary is formed. The adhesive path region and the substrate region are redefined based on the corrected adhesive path boundary. The local substrate reference surface is fitted using reliable three-dimensional points in the substrate region. The adhesive path height is redefined based on the normal distance of the three-dimensional points in the adhesive path region relative to the local substrate reference surface. The adhesive path boundary is then corrected again based on the redefined adhesive path height until the positional change between two adjacent adhesive path boundaries meets the preset convergence condition.
[0017] Preferably, in step S6: The longitudinal direction of the adhesive path is taken as the extension direction of the robot's adhesive application trajectory, the normal direction of the local substrate reference surface is taken as the height direction of the adhesive path, and the transverse direction of the adhesive path is determined based on the longitudinal direction and the height direction of the adhesive path. Multiple adhesive path cross sections are established along the longitudinal direction of the adhesive path. The width of the adhesive path is determined based on the corrected adhesive path boundary. The height of the adhesive path is determined based on the normal distance of the adhesive path surface relative to the local substrate reference surface. The cross-sectional area of the adhesive path is determined based on the height distribution of the adhesive path along the transverse direction. The coating volume of the corresponding adhesive segment is determined based on the cross-sectional area of the continuous adhesive path. Determine whether there is glue leakage or glue breakage based on the missing state and length of the glue path cross section; determine the glue path offset based on the positional deviation between the actual glue path center trajectory and the preset glue path center trajectory; determine whether there is excessive or insufficient glue application based on the difference between the glue path height, glue path cross-sectional area and / or glue volume and the corresponding preset parameters; and determine the uneven glue application based on the changes in width, height and / or cross-sectional area between adjacent glue path cross sections.
[0018] The second objective of this invention is to provide a three-dimensional fusion detection system for dynamic phase shift of a robot applying adhesive, comprising a detection component, a robot pose acquisition unit, an embedded processing unit, a historical state storage unit, and a defect output unit. The detection component is located at the end of the robot and includes multiple cameras with overlapping detection fields of view and a phase-shifted stripe projection unit, which is used to project phase-shifted stripes with different set phase shift amounts and simultaneously acquire corresponding phase-shifted images as the robot moves along the adhesive application trajectory. The robot pose acquisition unit is used to acquire the robot end pose corresponding to the exposure time of each phase-shift image. The embedded processing unit is used to perform spatial registration of phase-shifted images of the same phase-shifting period based on the robot end-effector pose and the predicted 3D surface; determine the projection geometric phase change and effective phase shift based on the spatial pose of the phase-shifting stripe projection unit at each exposure time; and analyze the wrapping phase and 2D texture image based on the effective phase shift. It is also used to generate candidate phase levels based on pre-stored workpiece 3D reference information and / or historical 3D adhesive path data in the historical state storage unit; determine the target phase level and 3D adhesive path data through the periodic phase consistency of candidate 3D points in other cameras; and perform closed-loop calibration of adhesive path boundaries and local substrate reference surfaces based on the 2D texture image and the 3D adhesive path data. The historical state storage unit is used to store reliable three-dimensional adhesive path data obtained during continuous detection. The defect output unit is used to determine and output adhesive path defect information based on the adhesive path boundary and three-dimensional adhesive path data after closed-loop calibration.
[0019] Preferably, the plurality of cameras includes three global shutter area array cameras; The embedded processing unit includes an ARM processing unit and an FPGA processing unit. The ARM processing unit is used to determine the image remapping parameters and effective phase shift parameters based on the robot end pose, the predicted three-dimensional surface and the spatial calibration relationship, and to perform candidate phase order generation, multi-view periodic phase consistency verification, three-dimensional data fusion, 2D-3D closed-loop calibration and adhesive path defect judgment. The FPGA processing unit is used to synchronously trigger and buffer phase-shifted images for the multiple cameras, remap the phase-shifted images according to the image remapping parameters, and perform dynamic phase analysis according to the effective phase shift parameters to obtain wrapping phase, common grayscale components and phase quality data. The historical state storage unit is also used to store the absolute phase, the corrected glue path boundary, and the corresponding robot trajectory information.
[0020] The beneficial effects of this invention are: This invention addresses the practical situation where the robot end effector continuously moves within the phase shift period. It not only performs spatial remapping of different phase shift images based on the robot's pose, but also calculates the projection geometric phase change of the same workpiece spatial point based on the spatial pose of the phase shift fringe projection unit at each exposure time. This change is then combined with the actively set phase shift amount to form an effective phase shift amount, thereby establishing dynamic phase analysis based on the actual motion geometry of the robot.
[0021] This invention utilizes historical 3D state or workpiece 3D reference information to form candidate phase levels. Then, it reads the current measured wrapping phase by the projection position of the candidate 3D points in other cameras and compares it with the theoretical projection phase corresponding to the candidate 3D points. This forms a level verification mechanism that combines time prediction with the current multi-view measured phase, which can reduce phase period misjudgment.
[0022] This invention obtains common grayscale components simultaneously during dynamic phase analysis, ensuring that two-dimensional texture information and three-dimensional phase data originate from the same camera, the same detection area, and the same motion compensation process. The two-dimensional initial boundary constrains the three-dimensional region, and the three-dimensional height gradient corrects the two-dimensional boundary in reverse. The corrected boundary is then used to re-establish the substrate reference surface and adhesive path height, which can reduce the propagation of boundary errors to the calculation of adhesive width, adhesive height, cross-sectional area, and volume.
[0023] This invention is suitable for implementation on an embedded heterogeneous platform composed of ARM and FPGA. It can separate image-level pipelined computation from robot pose, spatial geometry and cross-cycle state calculation, which is beneficial for realizing online detection under continuous robot motion conditions. Attached Figure Description
[0024] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a system block diagram of the detection system of the present invention; Figure 2 This is a flowchart of the detection method of the present invention; Figure 3 This is a schematic diagram of image spatial motion compensation and projection geometric phase compensation for the same phase shift period according to the present invention; Figure 4 This is a schematic diagram of the multi-view periodic phase consistency verification of the candidate phase order of the present invention; Figure 5 This is a schematic diagram of the two-dimensional texture and three-dimensional height closed-loop calibration of the present invention. Detailed Implementation
[0025] The present invention will be further described below with reference to specific embodiments. It should be understood that the following embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit the scope of protection of the present invention. Without departing from the concept of the present invention, adjustments to the number of cameras, stripe parameters, threshold parameters, replacement of processing modules, or equivalent data processing methods made by those skilled in the art based on the contents of this specification will not affect the implementation of the technical solutions of the present invention.
[0026] Example 1 like Figures 2 to 5 As shown, this embodiment provides a three-dimensional fusion detection method for dynamic phase shift of adhesive application in robots, which is applied to the automatic adhesive application station of robots. The detected adhesive path can be automotive body edge adhesive, structural adhesive, sealant, or other adhesives continuously formed along a predetermined trajectory.
[0027] like Figure 2 As shown, the three-dimensional fusion detection method for dynamic phase shift during adhesive application in this robot includes the following steps: S1. Multiple cameras with overlapping detection fields of view and a phase-shift fringe projection unit are set at the end of the robot to establish a spatial calibration relationship between the multiple cameras, the phase-shift fringe projection unit and the end of the robot; during the movement of the robot along the adhesive application trajectory, multiple phase-shift fringes with different set phase shift amounts are sequentially projected by the phase-shift fringe projection unit, and the corresponding phase-shift images are synchronously acquired by the multiple cameras, and the robot end pose corresponding to the exposure time of each phase-shift image is obtained.
[0028] Specifically, a detection assembly is installed at the end effector of the robot near the glue gun. The detection assembly includes a phase-shift fringe projection unit and three global shutter area array cameras. The three cameras are spaced apart circumferentially along the detection assembly, preferably approximately 120° apart in circumferential orientation (120° represents the circumferential distribution of the installation positions, and does not require the actual optical axes of the three cameras to form a strict 120° angle). Each camera faces the glue path detection area behind the direction of glue gun movement, forming at least partially overlapping detection fields of view.
[0029] The phase-shifting fringe projection unit can employ a digital light processing projection device, a liquid crystal spatial light modulation device, or other projection components capable of generating periodic fringes and changing the fringe phase according to control commands. After the robot completes the glue application at its current position, the newly formed glue path enters the detection area located behind the glue gun, enabling continuous glue application and detection.
[0030] Establish a robot base coordinate system B, a robot end effector coordinate system E, camera coordinate systems C1, C2, and C3 corresponding to the three cameras, a projection coordinate system P corresponding to the phase-shifting fringe projection unit, and a workpiece coordinate system W for the workpiece to be inspected. Historical 3D adhesive path data is preferably stored uniformly in either the robot base coordinate system B or the workpiece coordinate system W, ensuring that the actual positions of historical adhesive path points in a unified spatial coordinate system remain unchanged during robot end effector movement.
[0031] The internal parameters of the three cameras, such as focal length, principal point coordinates, and lens distortion parameters, were calibrated. The fixed spatial transformation relationship of each camera relative to the robot's end effector was obtained through hand-eye calibration. The spatial transformation relationship of the phase-shifting fringe projection unit relative to the robot's end effector was also calibrated. Furthermore, a mapping relationship was established between the three-dimensional spatial position and the projection coordinates and projection phase of the phase-shifting fringe projection unit.
[0032] One specific calibration method is to equate the phase-shifting fringe projection unit to an inverse camera model. For a spatial point X, it is transformed to the projection coordinate system P to obtain the projection coordinates. .
[0033] For a variable along the horizontal direction of the projection unit with a spatial period of The projected fundamental phase of the fringes in the zero active phase shift state can be expressed as: ; For a variable along the vertical direction of the projection unit with a spatial period of The projected fundamental phase of the fringes can be expressed as: .
[0034] in, and These are the projection coordinates after distortion correction by the projection unit.
[0035] In addition to using the geometric model mentioned above, a lookup model between camera pixels, projection phase and 3D position can also be established using calibration planes located at multiple known depth positions. As long as the corresponding projection phase can be determined based on the known 3D position and the 3D position can be recovered from the camera pixels and absolute phase, it is acceptable.
[0036] During the adhesive application process, the robot controller continuously outputs the position, orientation, and corresponding time information of the robot's end effector. Each phase-shifted image records the exposure timestamp. The camera exposure trigger signal and the robot pose data use a unified clock, or the time deviation between the two is pre-calibrated. The robot at any given time... The end pose is denoted as When the robot pose sampling time is not completely consistent with the midpoint of the camera exposure time, interpolation is performed based on the robot pose before and after the exposure. The position component can be interpolated linearly, and the rotational attitude can be interpolated using quaternion spherical interpolation.
[0037] The phase-shifting fringe projection unit projects periodic sinusoidal fringes along the first fringe direction and the second fringe direction, respectively. The first fringe direction and the second fringe direction are preferably perpendicular or substantially perpendicular to each other. Each fringe direction preferably employs four phase-shifting fringes, with the phase shift amount actively set. , , and When the values are 0, π / 2, π, and 3π / 2, and four images are projected in the first and second directions respectively, a complete acquisition group includes eight stripe states.
[0038] It should be noted that the above-mentioned actively set phase shift amount only represents the phase offset actively applied by the projection controller, and is not equivalent to the actual effective phase shift experienced by a fixed spatial point in the robot's motion state.
[0039] S2. For multiple phase-shifted images acquired within the same phase-shift period, determine the reference exposure time. Based on the robot end-effector pose at the exposure time of each phase-shifted image and the predicted three-dimensional surface of the current adhesive path detection area, perform image spatial motion compensation. Based on the spatial pose of the phase-shifted stripe projection unit at each exposure time, perform projection geometric phase motion compensation to obtain the effective phase shift amount corresponding to each phase-shifted image.
[0040] Specifically, the first step is to construct a predicted 3D surface for the current adhesive path detection area. Pre-stored 3D reference information for the workpiece can include the workpiece's CAD 3D model, pre-collected 3D point clouds of the substrate in a glue-free state, and substrate reference surfaces, among other things. In the initial detection phase, the predicted 3D surface can be directly constructed based on the pre-stored workpiece 3D reference information. Upon entering the continuous detection phase, the predicted 3D surface is further updated using reliable 3D adhesive path points stored in a unified spatial coordinate system from the previous detection cycle.
[0041] like Figure 3 As shown, for a phase shift period in the same fringe direction, one of the exposure times is selected as the reference exposure time. For predicting a spatial point X on a 3D surface, X is projected onto the nth image based on the camera's spatial pose at the exposure time of the nth phase-shifted image to obtain the pixel position. Based on the camera's spatial pose at the reference exposure time, the same spatial point X is projected onto the reference image to obtain the reference pixel position. Therefore, the pixel displacement is obtained:
[0042] The above calculations are performed on multiple spatial points on the predicted 3D surface to obtain the pixel displacement field in the region of interest (ROI). Based on this pixel displacement field, a remapping is performed on the non-reference phase-shifted image, so that the remapped phase-shifted images correspond to the same workpiece spatial region in the prediction sense at the same reference pixel. Preferably, the remapping is performed only on the ROI formed by the adhesive path and the substrates on both sides of the adhesive path to reduce the embedded computational load.
[0043] After pixel position registration is completed, the phase change of the projection base caused by the motion of the phase-shifting fringe projection unit is further compensated. Based on the robot end-effector pose at the exposure time of the nth image and the installation and calibration relationship of the phase-shifting fringe projection unit, the fixed spatial point X is transformed to the projection coordinate system at that time to obtain the projection coordinates. .
[0044] Taking the direction of the first fringe as an example, the projection base phase of the nth exposure is determined based on the projection coordinates, without considering the active phase shift. The projection base phase at the reference exposure time is Then the geometric phase change is:
[0045] Let the set phase shift amount actively applied by the controller for the nth phase shift fringe be... The effective phase shift of a fixed spatial point X relative to the reference geometric state is... for:
[0046] The effective phase shift can be calculated in the same way for the second fringe direction. Since subsequent dynamic phase analysis mainly uses... and Therefore, the effective phase shift can be a continuous phase value or folded back to a 2π period. After the above processing, it is not necessary to assume that the actual phase change of a fixed spatial point during the robot's motion is still strictly equal to 0, π / 2, π, and 3π / 2.
[0047] Furthermore, the first spatial motion compensation and geometric phase compensation use a predicted 3D surface. After obtaining the preliminary 3D adhesive path data for the current cycle, the spatial deviation between it and the predicted 3D surface can be calculated. When the deviation exceeds a preset reconstruction deviation threshold, the predicted 3D surface is updated with the currently obtained high-reliability 3D points, and the pixel displacement and effective phase shift are recalculated to reduce the residual dynamic compensation error caused by the difference between the predicted surface and the real adhesive surface.
[0048] S3. Based on the effective phase shift amount described in S2, perform dynamic phase analysis on the registered multiple phase-shifted images to obtain the wrapping phase, fringe modulation and common gray-scale components, and form a two-dimensional texture image based on the common gray-scale components.
[0049] Specifically, for the same reference pixel after spatial remapping, let the gray level corresponding to the nth phase-shifted image be... Its effective phase shift is The package phase under the reference exposure state is denoted as... Let B be the amplitude of the fringe modulation, and A be the common grayscale component that is independent of the phase shift state. Then the grayscale can be expressed as:
[0050] Expand the cosine term and let , Then we get:
[0051] For N phase-shifted images in the same fringe direction, all grayscale equations can be written as a system of linear equations. When there are at least three different effective phase shifts and the corresponding coefficient matrices satisfy the full-rank condition, A, C, and S can be solved. Preferably, four or more images are used, and the solution is obtained using the least squares method. After obtaining C and S, the wrap-around phase under the reference exposure state is used. The stripe tone M is as follows:
[0052]
[0053] When the robot moves very little within a phase shift cycle, causing a change in geometric phase... When approximately zero, the effective phase shift Approximately equal to the actively set phase shift. The above general dynamic phase solution can be degenerated into a conventional four-step phase shift solution with equal intervals.
[0054] The A obtained during the dynamic phase calculation is the common grayscale component independent of the phase shift state. Since A is obtained by jointly solving for the actual effective phase shift of each image, it can still be used to form a two-dimensional textured image that removes the effects of periodic fringe modulation, even if the robot's movement causes the actual phase shift to no longer be strictly equal. A is calculated for each pixel within the ROI to form a two-dimensional textured image corresponding to the fringe direction; the common grayscale images obtained in the first and second fringe directions can be averaged and fused, or the one with higher quality can be selected based on phase quality.
[0055] Furthermore, the quality of the wrapped phase is evaluated. Based on the obtained A, C, S, and known effective phase shift, the theoretical gray level of the nth image is calculated. And calculate the phase fitting residual based on the difference between the actual gray level and the theoretical gray level. For example, it can be done as follows:
[0056] Phase quality is determined at least based on the fringe modulation degree M and the fitted residual. The condition number of the solution matrix can be determined by combining pixel overexposure, underexposure, and effective phase shift. If the stripe modulation is too low, the fitting residual is too large, or the solution matrix is obviously ill-conditioned, the corresponding position is marked as a low-reliability phase pixel.
[0057] S4. Generate candidate phase levels corresponding to the current package phase based on the state of the detection cycle.
[0058] Specifically, such as Figure 4 As shown, the dynamic phase analysis yields... The wrap-around phase within a 2π period, absolute phase It can be represented as: , where k is an integer phase order.
[0059] When there is no historical reliable 3D adhesive path data available at the start of the inspection cycle, the substrate surface position of the current inspection area is obtained based on the pre-stored 3D reference information of the workpiece, and combined with the maximum allowable height corresponding to the adhesive path process. and the minimum allowable height when necessary. This process forms a candidate three-dimensional spatial range where the colloid may exist. For the pixel to be measured by the camera, the possible three-dimensional positions within the candidate spatial range are determined using the camera observation rays. Based on the geometric relationship between these three-dimensional positions and the phase-shifted fringe projection units under the reference exposure state, the possible phase orders are determined, forming an initial candidate order set.
[0060] After the initial inspection cycle is completed, the reliable 3D adhesive path points are uniformly converted and stored in the robot base coordinate system or the workpiece coordinate system. In subsequent inspection cycles, the actual positions of the historical 3D points in the unified spatial coordinate system are not changed. Instead, based on the current robot end-effector pose and the installation and calibration relationship of the phase-shift fringe projection unit, the historical 3D points are converted and represented in the projection coordinate system of the current reference exposure time, thereby calculating their corresponding current predicted absolute phase. For the currently measured package phase Predicting phase order It can be determined according to the following formula:
[0061] Considering historical changes in adhesive path morphology, robot positioning errors, and current local variations in adhesive volume, we do not directly... As the final level, and with Construct a finite set of candidate levels for the center, for example For newly entered locations in the current field of view that are not directly covered by historical 3D points, predictive constraints can be formed by combining the continuous variation trend of adjacent historical adhesive path sections along the robot trajectory direction with the 3D reference information of the workpiece substrate; if a reliable historical prediction still cannot be formed, then the candidate depth range constraint of the initial detection cycle is re-adopted.
[0062] S5. Perform multi-view periodic phase consistency verification on the candidate phase levels described in S4, determine the target phase level, and restore the three-dimensional adhesive path data.
[0063] Specifically, such as Figure 4 As shown, for any candidate level in the candidate level set According to the current package phase Candidate absolute phases are obtained: .
[0064] By combining the current reference camera pixel position, candidate absolute phase, and pre-established camera-projection unit spatial calibration model, the corresponding candidate 3D points are recovered. Then, the validity of the candidate 3D point is verified by using the package phase actually measured by other cameras during the current detection period.
[0065] Taking the first camera as the main computational viewpoint for candidate 3D points and the second camera as the verification viewpoint as an example, the candidate 3D points are... The spatial pose of the second camera at the reference exposure time is projected onto the second camera image coordinate system to obtain the projected position. Theoretically, the projection phase corresponding to a candidate 3D point is determined by the spatial relationship of that 3D point relative to the phase-shifting fringe projection unit, rather than by the second camera. Therefore, according to Based on the spatial pose of the phase-shifted fringe projection unit at the reference exposure time, calculate the predicted projection absolute phase corresponding to the candidate 3D point, and then fold it back to a 2π period to obtain the predicted wrap-around phase. On the other hand, from the package phase image already parsed from the current detection period of the second camera, in Location reading measured package phase .
[0066] The periodic phase residual between the predicted package phase and the measured package phase can be expressed as:
[0067] Similarly, the periodic phase residual corresponding to the third camera can be calculated. When both the second and third cameras have valid phase data, a composite periodic phase residual can be generated according to the reliability of the corresponding viewing angle: ,in Candidate phase levels whose integrated periodic phase residuals satisfy preset consistency conditions are selected as target phase levels. When multiple candidate levels meet the basic consistency requirements, the final target level can be determined by further considering the distance between candidate 3D points and historical 3D prediction constraints, the allowable height range of the adhesive path, and the 3D continuity of the neighborhood.
[0068] Furthermore, the reliability of the 3D adhesive path data obtained from multiple cameras is determined. The basic reliability is determined at least based on the fringe modulation and multi-view periodic phase consistency, and then corrected based on pixel saturation, dynamic phase fitting residuals, deviations between 3D points and historical prediction constraints, and neighborhood continuity. For the same spatial region, when the spatial deviation between the 3D results obtained from multiple views does not exceed a preset consistency threshold, reliability-weighted fusion can be performed; when the spatial deviation exceeds the consistency threshold, simple averaging is not performed, and results with higher reliability are retained first.
[0069] For example, two three-dimensional points are respectively and The corresponding reliability levels are respectively and At the same time, fusion of three-dimensional points It can be represented as:
[0070] S6. Perform 2D-3D closed-loop calibration based on the 2D texture image and 3D adhesive path data to determine the corrected adhesive path boundary and local substrate reference surface.
[0071] Specifically, such as Figure 5 As shown, the intermediate camera or the camera with the highest current phase quality is selected as the reference viewpoint for 2D-3D fusion. Reliable 3D points obtained from other cameras are uniformly transformed to the robot base coordinate system or workpiece coordinate system, and then reprojected to the reference viewpoint as needed. Candidate edges between the colloid and the substrate are identified based on grayscale gradients, color differences, or local contrast in the 2D texture image. These edges are then combined with the robot's preset adhesive application trajectory to remove isolated edges that significantly deviate from the theoretical adhesive path position, thus obtaining the initial adhesive path boundary. .
[0072] Utilizing the initial adhesive path boundary The three-dimensional points were initially divided into candidate regions for adhesive paths and candidate regions for substrates, and then along... The edge search area is formed by extending inwards and outwards towards the adhesive path. For Boundary points on The local tangent direction is determined based on adjacent boundary points, and the boundary normal direction, which is basically perpendicular to the local tangent, is also determined. Multiple candidate boundary locations are set within the edge search area along the boundary normal direction. .
[0073] For each candidate boundary position q, calculate the two-dimensional texture gradient. and 3D height gradient Specifically, the 3D height gradient is preferably calculated relative to the local substrate trend surface, rather than directly using the Z-axis depth change in the camera coordinate system, to avoid the spatial changes of the inclined or curved substrate itself being mistaken for the adhesive path height boundary. Further, the reliability of the 3D data corresponding to the candidate locations is obtained. and the continuity deviation of the candidate position relative to the adjacent boundary trend .
[0074] In one specific embodiment, the following comprehensive evaluation metric is used to determine the candidate boundary position:
[0075] in, and These are the normalized 2D texture gradient and 3D height gradient, respectively. , , and To evaluate the weights, each weight can be pre-calibrated using standard adhesive sample. In areas where there is a significant color difference between the adhesive and the substrate but strong local reflectivity in the metallic substrate, the influence of the two-dimensional texture gradient can be enhanced; in areas where the texture of the transparent adhesive or the adhesive is similar to that of the substrate but where there are significant changes in three-dimensional height, the influence of the three-dimensional height gradient can be enhanced.
[0076] Based on the comprehensive evaluation, the location of the correction boundary is determined and connected to form the first correction adhesive path boundary. ,according to Re-divide the adhesive path area and the substrate area, from Three-dimensional points with acceptable reliability are selected from the outer substrate area. When the local curvature of the substrate is low, planar fitting is used; when the substrate has significant curvature, local quadratic surface fitting is used. Alternatively, the position is corrected by combining the workpiece CAD local surface with measured substrate points to obtain a local substrate reference surface. .
[0077] For three-dimensional points within the adhesive path area Calculate its relative to the local substrate reference plane. The normal distance is calculated and used as the actual height of the adhesive path. After obtaining the adhesive path height again, the three-dimensional height gradient in the edge search region is calculated and re-evaluated with the two-dimensional texture gradient to obtain the second corrected adhesive path boundary. .calculate and The average position change between the points is calculated. When the change is less than a preset convergence threshold, boundary correction is stopped. When the change still exceeds the preset convergence threshold, the base material point is reselected according to the current boundary, the local base material reference surface is updated, and the boundary is corrected again until the convergence condition is met or the preset maximum number of iterations is reached.
[0078] S7. Establish a local coordinate system for the adhesive path based on the robot's adhesive application trajectory and the local substrate reference surface, and calculate the three-dimensional cross-sectional parameters of the adhesive path.
[0079] Specifically, regarding the position on the robot's adhesive application trajectory Determine the longitudinal unit direction of the adhesive path based on adjacent trajectory points. The normal unit direction is determined based on the local substrate reference plane. ,Will The height direction of the adhesive path is used as the reference direction, and the lateral direction of the adhesive path is determined according to the following formula. :
[0080] Depend on , and A local coordinate system is constructed that varies with the adhesive path trajectory. Multiple adhesive path cross-sections are established at predetermined intervals along the longitudinal direction of the adhesive path. The width of the adhesive path is determined based on the intersection of the current cross-section and the corrected left and right adhesive path boundaries. The height of the adhesive path is determined based on the normal distance of the three-dimensional points on the adhesive surface relative to the local substrate reference surface.
[0081] Based on the height distributed along the transverse direction of the adhesive path Calculate the cross-sectional area of the adhesive path :
[0082] in, and These represent the left and right boundary positions of the adhesive path at the current cross-section. For discrete three-dimensional points, numerical methods such as the trapezoidal integral can be used for calculation. Further calculations are performed along the longitudinal direction of the adhesive path:
[0083] Obtain the trajectory interval to The volume V of the colloid within it. Therefore, the location, width, height, cross-sectional area of the adhesive path, and the volume of the colloid in a given length of adhesive segment can be obtained simultaneously.
[0084] S8. Determine the glue path defect information based on the difference between the glue path boundary and three-dimensional cross-sectional parameters and the preset glue path parameters.
[0085] Specifically, based on the current vehicle model and workstation, the standard adhesive path center trajectory, adhesive path width range, height range, cross-sectional area range, adhesive volume per unit length range, and allowable continuous missing length are pre-stored. Adhesive leakage or interruption is determined by the presence of valid three-dimensional adhesive data along the longitudinal continuous cross-section of the adhesive path and the corresponding missing length; adhesive path offset is determined by the lateral positional deviation between the actual adhesive path center trajectory and the standard adhesive path center trajectory; insufficient or excessive adhesive application is determined by the relationship between the actual adhesive path height, cross-sectional area, or adhesive volume per unit length and the standard range; and uneven adhesive application is determined by the degree of variation in width, height, or cross-sectional area between continuous adhesive path sections.
[0086] The above defect determination can be directly achieved by comparing geometric parameters with process thresholds, without requiring the use of a machine learning classification model. Each preset threshold can be pre-calibrated based on standard defect-free adhesive sample and known defect sample for the corresponding vehicle model and workstation.
[0087] For example, the normal detection range is determined by repeatedly testing the statistical stripe modulation, phase fitting residual, multi-view periodic phase residual, and boundary repetition positioning error of standard adhesive samples; then, the corresponding defect judgment threshold is confirmed by using samples with known adhesive breaks, offsets, insufficient adhesive, or excessive adhesive.
[0088] Furthermore, when arc light or other transient strong light exists in the welding workshop, the actual grayscale of the local phase-shifted image will deviate from the dynamic sine model, manifesting as an increase in phase fitting residuals, a sudden increase in local grayscale, or pixel saturation. When a local area of a camera is interfered with by transient light, reliable 3D data obtained by other cameras in the same spatial area is prioritized. When all viewpoints lack reliable measured data in the current inspection cycle, historical 3D states can be used to maintain the prediction constraints for the next cycle, but historical prediction results are not directly used as high-confidence normal judgments for the current cycle. When reliable measured data cannot be obtained for several consecutive cycles, the corresponding adhesive segment is marked as an area to be re-inspected.
[0089] Furthermore, the subsequent phase-shift fringe parameters can be adjusted based on the phase quality of the current detection area. When the proportion of locally saturated pixels is high, the projection brightness is reduced; when there is no obvious saturation but the overall fringe modulation is low, the projection brightness is increased. When the active phase shift and the projection geometric phase change are superimposed, causing multiple effective phase shifts to be too close and the dynamic phase solution matrix conditions to deteriorate, the subsequent active phase shift is adjusted to maintain good solution conditions for the effective phase shift combination.
[0090] Example 2 like Figure 1As shown, this embodiment provides a three-dimensional fusion detection system for dynamic phase shift of a robot applying adhesive, used to execute the method described in Embodiment 1. The system includes a detection component, a robot pose acquisition unit, an embedded processing unit, a historical state storage unit, and a defect output unit.
[0091] The detection assembly is mounted on the end effector of the robot and moves synchronously with the glue applicator. The assembly includes three global shutter area array cameras and a phase-shift fringe projection unit. The three cameras have overlapping fields of view covering the glue path area behind the glue applicator. The phase-shift fringe projection unit projects periodic fringes with different directions, different set phase shifts, and different spatial periods according to a control sequence.
[0092] The robot pose acquisition unit communicates with the robot controller, continuously acquiring the robot's end-effector position, posture, and corresponding timestamps, and then sends the pose data to the embedded processing unit. A unified time relationship is established between the camera exposure timestamp and the robot pose timestamp.
[0093] The embedded processing unit employs a heterogeneous computing platform composed of ARM and FPGA processing units. The ARM processing unit is primarily responsible for calculations related to the robot's motion state, three-dimensional spatial geometric relationships, and historical states across detection cycles; the FPGA processing unit is primarily responsible for multi-camera synchronization triggering, image caching, ROI remapping, solving dynamic phase equations, and pixel-level phase quality calculations.
[0094] After a phase shift cycle begins, the ARM processing unit constructs a predicted 3D surface based on reliable 3D adhesive path data and / or pre-stored 3D reference information of the workpiece in the historical state storage unit. It calculates the image remapping parameters within the ROI based on the robot pose and camera calibration relationship corresponding to each phase shift image. At the same time, it calculates the projection geometric phase change based on the spatial pose of the phase shift stripe projection unit at each exposure time, and generates the effective phase shift parameters for each pixel by combining the actively set phase shift amount.
[0095] The ARM processing unit sends the image remapping parameters and effective phase shift parameters to the FPGA processing unit. The FPGA processing unit receives and buffers the phase-shifted images simultaneously acquired by the three cameras, performs ROI-level remapping on the phase-shifted images of the same phase shift period according to the remapping parameters, and establishes a model based on the effective phase shift. The dynamic phase-shift model is used to obtain A, C, and S in a pipeline manner, and outputs the wrapping phase, common gray-scale components, stripe modulation, fitting residuals, and pixel saturation state.
[0096] The common grayscale component is used to form a two-dimensional texture image, the wrapping phase is used for subsequent phase order verification, and the stripe modulation, fitting residual, and pixel saturation state are used to form phase quality data. The above results are sent to the ARM processing unit.
[0097] For the initial detection cycle, the ARM processing unit generates candidate phase levels based on the workpiece's three-dimensional reference information and the allowable height range of the adhesive path. For subsequent detection cycles, the ARM processing unit reads reliable three-dimensional adhesive path data from the historical state storage unit in a unified spatial coordinate system, determines the predicted absolute phase based on the current projection unit's spatial pose, and generates candidate phase levels near the predicted level.
[0098] The ARM processing unit recovers candidate 3D points for each candidate level, projects the candidate 3D points onto other cameras, and calculates the predicted wrapping phase based on the spatial relationship between the candidate 3D points and the phase-shifting fringe projection unit under the reference exposure state. It reads the measured wrapping phase at the corresponding projection position from the wrapping phase map of other cameras, calculates the periodic residual between the predicted wrapping phase and the measured wrapping phase, and thus determines the target phase level and the final 3D adhesive path point.
[0099] After completing the 3D reconstruction, the ARM processing unit extracts the initial adhesive path boundary from the 2D texture image and projects reliable 3D points onto a unified reference viewpoint. The initial adhesive path boundary is used to divide the adhesive path region and the substrate region. Search regions are established on both sides of the boundary, and the correction boundary is determined by combining the 2D texture gradient and the 3D height gradient. Based on the correction boundary, 3D points on the substrate are reselected and a local substrate reference surface is fitted. The boundary correction is then repeated based on the updated adhesive path height until the convergence condition is met.
[0100] The historical state storage unit stores at least the reliable 3D adhesive path data in the robot base coordinate system or workpiece coordinate system, the determined absolute phase, the corrected adhesive path boundary, the corresponding robot trajectory position, and the corresponding data reliability. After the current inspection cycle is completed, the high-reliability result is written to the historical state storage unit for the next inspection cycle to construct the predicted 3D surface and generate the absolute phase prediction.
[0101] The defect output unit communicates with the robot controller and / or the production line PLC. The ARM processing unit generates a defect judgment result based on parameters such as position, width, height, cross-sectional area, and volume in the local coordinate system of the glue path, and outputs alarms, line stop, rework markings, or glue replenishment commands through the defect output unit. When the production control system supports online robot correction, the defect spatial position can also be converted to the robot's execution coordinate system, providing position parameters for the robot's subsequent glue replenishment actions.
[0102] When the system runs continuously, within a single phase shift cycle, the robot pose acquisition, image spatial motion compensation, projection geometric phase compensation, and dynamic phase resolution are performed sequentially. Between adjacent detection cycles, feedback is provided by historical 3D states to predict the current phase and re-verify the current multi-view measured phase. After 3D reconstruction, a 2D-3D closed-loop operation is achieved, which defines the 3D region by 2D boundaries, corrects the 2D boundaries by 3D height, and updates the substrate reference surface by correcting the boundaries.
[0103] The modules described in the above embodiments can be implemented using software, hardware, or a combination of both. Each step can be adaptively parallelized while ensuring data dependencies and technical effectiveness. The number of cameras, the number of stripes, and the specific thresholds mentioned above are provided for the purpose of illustrating the feasible implementations of the present invention and do not constitute a limitation on the scope of protection of the present invention.
[0104] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A three-dimensional fusion detection method for dynamic phase shift in robot adhesive application, characterized in that, Includes the following steps: S1. Set multiple cameras with overlapping detection fields of view and a phase-shifting fringe projection unit at the end of the robot, and establish the spatial calibration relationship between the multiple cameras, the phase-shifting fringe projection unit and the end of the robot; During the robot's movement along the adhesive application trajectory, the phase-shifting stripe projection unit sequentially projects multiple phase-shifting stripes with different set phase shift amounts. The multiple cameras synchronously acquire the corresponding phase-shifting images and obtain the robot's end-effector pose corresponding to the exposure time of each phase-shifting image. S2. Based on the robot end-effector pose at each phase-shift image exposure time and the predicted three-dimensional surface determined by at least one of the pre-stored workpiece three-dimensional reference information and the three-dimensional adhesive path data from the previous detection cycle, multiple phase-shift images within the same phase-shift cycle are registered to the reference exposure time; based on the spatial pose of the phase-shift stripe projection unit at each exposure time, the projection geometric phase change of the same spatial position relative to the reference exposure time is determined, and the projection geometric phase change is combined with the set phase shift amount of the corresponding phase-shift stripe to obtain the effective phase shift amount; S3. Perform dynamic phase analysis on the registered multiple phase-shifted images according to the effective phase shift amount to obtain the wrapping phase under the reference exposure state and the common gray-scale component that is independent of the phase shift state, and form a two-dimensional texture image of the adhesive path detection area according to the common gray-scale component. S4. Establish phase order prediction constraints for the wrapping phase based on at least one of the pre-stored workpiece 3D reference information and historical 3D adhesive path data, and generate candidate phase orders accordingly; recover candidate 3D points based on each candidate phase order, project the candidate 3D points onto at least one other camera, and determine the target phase order based on the periodic phase residual between the predicted wrapping phase corresponding to the candidate 3D point and the measured wrapping phase at the corresponding projection position of the other camera, so as to obtain 3D adhesive path data of the adhesive path detection area. S5. Obtain the initial adhesive path boundary based on the two-dimensional texture image. Correct the adhesive path boundary in the neighborhood of the initial adhesive path boundary by combining the two-dimensional texture gradient and the three-dimensional height gradient of the three-dimensional adhesive path data. Redetermine the adhesive path region and the substrate region based on the corrected adhesive path boundary. Construct a local substrate reference surface using the three-dimensional data of the substrate region. Determine the three-dimensional cross-sectional parameters of the adhesive path based on the local substrate reference surface. S6. Determine the glue path position and glue path geometric parameters based on the robot's glue application trajectory, the corrected glue path boundary, and the glue path three-dimensional cross-sectional parameters, and determine the glue path defect information based on the difference between the glue path position and glue path geometric parameters and the preset glue path parameters.
2. The three-dimensional fusion detection method for dynamic phase shift of robot adhesive application according to claim 1, characterized in that, The plurality of cameras includes three global shutter area array cameras, which are arranged circumferentially at intervals along the detection component at the end of the robot and facing the glue path detection area located behind the direction of movement of the glue gun. The internal parameters and lens distortion parameters of each global shutter array camera, the pose relationship of each global shutter array camera relative to the robot end effector, and the pose relationship of the phase shift fringe projection unit relative to the robot end effector are pre-calibrated. The robot end-effector pose data output by the robot controller is correlated with the phase-shifted image according to a unified time reference. When the sampling time of the robot end-effector pose data is inconsistent with the exposure time of the phase-shifted image, pose interpolation is performed based on the robot end-effector pose at adjacent sampling times.
3. The three-dimensional fusion detection method for dynamic phase shift of robot adhesive application according to claim 1, characterized in that, In step S2: The same spatial point on the predicted three-dimensional surface is projected onto each phase-shift image according to the camera spatial pose corresponding to the exposure time of each phase-shift image. The pixel displacement of the spatial point relative to the reference exposure time is determined, and the phase-shift image is remapped according to the pixel displacement. The spatial points are transformed to the corresponding projection coordinate system according to the spatial pose of the phase-shifting stripe projection unit at each exposure time. The projection base phase is determined according to its projection coordinates, and the projection geometric phase change is obtained based on the projection base phase at the reference exposure time. When the spatial deviation between the three-dimensional adhesive path data obtained in the current detection cycle and the predicted three-dimensional surface exceeds the preset reconstruction deviation threshold, the predicted three-dimensional surface is updated with the current three-dimensional adhesive path data, and the pixel displacement and effective phase shift are re-determined.
4. The three-dimensional fusion detection method for dynamic phase shift of robot adhesive application according to claim 1, characterized in that, In step S3: For the nth registered phase-shifted image corresponding to the same fringe direction, the gray value of the same reference pixel satisfy: Where A is a common grayscale component that is independent of the phase shift state. Let be the effective phase shift amount corresponding to the nth phase-shifted image, and C and S be parameters to be determined related to the wrapping phase and fringe modulation amplitude under the reference exposure state. Solve for A, C, and S based on at least three phase shift images with different effective phase shifts, and determine the wrap-around phase. : Stripe tone system M: ; The basic phase quality is determined based on the fringe modulation and phase fitting residual, and the basic phase quality is corrected based on the pixel saturation state.
5. The three-dimensional fusion detection method for dynamic phase shift of robot adhesive application according to claim 1, characterized in that, In step S4: In the initial detection cycle, a candidate three-dimensional spatial range is determined based on the pre-stored three-dimensional reference information of the workpiece and the allowable height range of the adhesive path, and a candidate phase level is determined based on the candidate three-dimensional spatial range; In subsequent detection cycles, the reliable three-dimensional adhesive path data obtained in the previous detection cycle is stored in the robot base coordinate system or the workpiece coordinate system. The predicted absolute phase corresponding to the reliable three-dimensional adhesive path data is determined according to the spatial pose of the phase shift fringe projection unit at the current reference exposure time. The predicted phase level is determined according to the predicted absolute phase and the current wrapping phase. The predicted phase level and its adjacent phase levels form a candidate level set. For each candidate phase level, candidate 3D points are recovered. The predicted wrapping phase is determined based on the spatial relationship between the candidate 3D points and the phase shift fringe projection unit at the reference exposure time. The candidate 3D points are then projected onto other cameras to read the measured wrapping phase at the corresponding positions. The target phase level is determined based on the periodic phase residual between the predicted wrapping phase and the measured wrapping phase.
6. The three-dimensional fusion detection method for dynamic phase shift of robot adhesive application according to claim 1, characterized in that, In step S4: For 3D adhesive path data obtained from different cameras, the basic reliability is determined based on the stripe modulation and multi-view periodic phase consistency, and the basic reliability is corrected based on one or more of the following: pixel saturation state, phase fitting residual, deviation between 3D position and historical 3D prediction data, and phase continuity between adjacent spatial points. When the spatial deviation between 3D data obtained from different cameras in the same spatial area does not exceed a preset consistency threshold, the 3D data are weighted and fused according to their corresponding reliability; when the spatial deviation exceeds the preset consistency threshold, the 3D data with higher reliability is retained.
7. The three-dimensional fusion detection method for dynamic phase shift of robot adhesive application according to claim 1, characterized in that, In step S5: An edge search region is formed by extending along the initial adhesive path boundary to the inner and outer sides of the adhesive path. Multiple candidate boundary positions are determined within the edge search region along the local normal direction of the initial adhesive path boundary. The corrected boundary position is determined based on the two-dimensional texture gradient, three-dimensional height gradient, three-dimensional data reliability, and boundary contour continuity of each candidate boundary position, and the corrected adhesive road boundary is formed. The adhesive path region and the substrate region are redefined based on the corrected adhesive path boundary. The local substrate reference surface is fitted using reliable three-dimensional points in the substrate region. The adhesive path height is redefined based on the normal distance of the three-dimensional points in the adhesive path region relative to the local substrate reference surface. The adhesive path boundary is then corrected again based on the redefined adhesive path height until the positional change between two adjacent adhesive path boundaries meets the preset convergence condition.
8. The three-dimensional fusion detection method for dynamic phase shift of robot adhesive application according to claim 1, characterized in that, In step S6: The longitudinal direction of the adhesive path is taken as the extension direction of the robot's adhesive application trajectory, the normal direction of the local substrate reference surface is taken as the height direction of the adhesive path, and the transverse direction of the adhesive path is determined based on the longitudinal direction and the height direction of the adhesive path. Multiple adhesive path cross sections are established along the longitudinal direction of the adhesive path. The width of the adhesive path is determined based on the corrected adhesive path boundary. The height of the adhesive path is determined based on the normal distance of the adhesive path surface relative to the local substrate reference surface. The cross-sectional area of the adhesive path is determined based on the height distribution of the adhesive path along the transverse direction. The coating volume of the corresponding adhesive segment is determined based on the cross-sectional area of the continuous adhesive path. Determine whether there is glue leakage or glue breakage based on the missing state and length of the glue path cross section; determine the glue path offset based on the positional deviation between the actual glue path center trajectory and the preset glue path center trajectory; determine whether there is excessive or insufficient glue application based on the difference between the glue path height, glue path cross-sectional area and / or glue volume and the corresponding preset parameters; and determine the uneven glue application based on the changes in width, height and / or cross-sectional area between adjacent glue path cross sections.
9. A three-dimensional fusion detection system for dynamic phase shift of a robot applying adhesive, characterized in that, It includes a detection component, a robot pose acquisition unit, an embedded processing unit, a historical state storage unit, and a defect output unit; The detection component is located at the end of the robot and includes multiple cameras with overlapping detection fields of view and a phase-shifted stripe projection unit, which is used to project phase-shifted stripes with different set phase shift amounts and simultaneously acquire corresponding phase-shifted images as the robot moves along the adhesive application trajectory. The robot pose acquisition unit is used to acquire the robot end pose corresponding to the exposure time of each phase-shift image. The embedded processing unit is used to perform spatial registration of phase-shifted images of the same phase-shifting period based on the robot end-effector pose and the predicted three-dimensional surface, determine the projection geometric phase change and effective phase shift based on the spatial pose of the phase-shifting stripe projection unit at each exposure time, and parse the wrapping phase and two-dimensional texture image based on the effective phase shift. It is also used to generate candidate phase levels based on pre-stored workpiece three-dimensional reference information and / or historical three-dimensional adhesive path data in the historical state storage unit, determine the target phase level and three-dimensional adhesive path data by the periodic phase consistency of candidate three-dimensional points in other cameras, and perform closed-loop calibration of adhesive path boundary and local substrate reference surface based on two-dimensional texture image and three-dimensional adhesive path data. The historical state storage unit is used to store reliable three-dimensional adhesive path data obtained during continuous detection. The defect output unit is used to determine and output adhesive path defect information based on the adhesive path boundary and three-dimensional adhesive path data after closed-loop calibration.
10. The robotic adhesive application dynamic phase shift three-dimensional fusion detection system according to claim 9, characterized in that, The multiple cameras include three global shutter area array cameras; The embedded processing unit includes an ARM processing unit and an FPGA processing unit. The ARM processing unit is used to determine the image remapping parameters and effective phase shift parameters based on the robot end pose, the predicted three-dimensional surface and the spatial calibration relationship, and to perform candidate phase order generation, multi-view periodic phase consistency verification, three-dimensional data fusion, 2D-3D closed-loop calibration and adhesive path defect judgment. The FPGA processing unit is used to synchronously trigger and buffer phase-shifted images for the multiple cameras, remap the phase-shifted images according to the image remapping parameters, and perform dynamic phase analysis according to the effective phase shift parameters to obtain wrapping phase, common grayscale components and phase quality data. The historical state storage unit is also used to store the absolute phase, the corrected glue path boundary, and the corresponding robot trajectory information.
