A gluing quality visual inspection method and system for a body-in-white welding line

Abstract

The application discloses a kind of for white body welding line's glue quality visual detection method and system, it is related to industrial vision detection technical field, including, after white body enters welding detection station, obtain station ID, serial number and white body model, establish PLC session and load expected camera set, receive the image of glue area that multi-camera acquisition and update arrived camera set, when arrived camera set covers expected camera set and does not time out, determine effective camera and trigger controlled parallel detection;In controlled parallel detection, the image of glue area of each effective camera is carried out brightness, contrast, definition and noise self-evaluation, and generates image quality self-evaluation result.The application is realized by PLC session, expected camera set and arrived camera set that the collection control of multi-camera acquisition and detection trigger is triggered, reduces the problem that image and determination result do not match.

Description

Technical Field

[0001] This invention relates to the field of industrial visual inspection technology, and in particular to a visual inspection method and system for adhesive coating quality in body-in-white welding lines. Background Technology

[0002] In the body-in-white welding production line, the quality of adhesive application directly affects the sealing, waterproofing, rust prevention, and structural strength of the body, making it a key quality control link in the vehicle manufacturing process. With the increasing automation of automotive welding production lines, adhesive application quality inspection is gradually shifting from manual visual inspection to industrial vision inspection. Existing solutions typically deploy multiple industrial cameras at the welding inspection station to acquire images of different adhesive application areas on the body-in-white. Through image segmentation, area measurement, and defect judgment, quality problems such as adhesive leakage, adhesive breakage, abnormal adhesive width, and abnormal adhesive length are identified to meet the high-speed, continuous production requirements of the assembly line.

[0003] However, existing methods have some shortcomings. In debugging mode, multi-camera detection tasks are usually executed serially, and the detection waiting time accumulates with the number of cameras. Furthermore, the efficiency of cross-directory retrieval and image group reuse of debugging images is low, resulting in a long parameter adjustment cycle. The output of geometric measurement data such as glue width and glue length lacks regional measurement execution markers, and some areas that have not actually completed measurement may also output invalid logs, affecting subsequent quality analysis. In multi-station, multi-camera collaborative detection scenarios, the timing between detection triggering, result writing back, and image saving is difficult to unify, which can easily lead to problems such as delayed detection results and mismatch between images and judgment results. At the same time, the white body welding site has complex working conditions such as uneven lighting, high reflectivity of glue path, shadow occlusion, and slight deviations in tooling positioning, which can easily cause glue path segmentation errors and detection area offsets. In addition, defect judgment mostly relies on a single area difference index, without fully combining glue path topological connectivity, width stability, and cross-view consistency of multiple cameras, resulting in insufficient accuracy in judging defects such as glue leakage, glue breakage, width fluctuation, and glue length exceeding limits, making it difficult to meet the high-cycle, high-reliability quality control requirements of the white body welding production line. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a visual inspection method for adhesive coating quality in body-in-white welding lines to solve the problems of unstable multi-camera collaborative control, poor adaptability to complex working conditions, and insufficient accuracy in defect judgment.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a visual inspection method for adhesive coating quality in a body-in-white welding line, comprising: after the body-in-white enters the welding inspection station, obtaining the station ID, serial number, and body-in-white model; establishing a PLC session and loading a desired camera set; receiving adhesive coating area images acquired by multiple cameras and updating the already arrived camera set; when the already arrived camera set covers the desired camera set and no timeout occurs, determining the effective cameras and triggering controlled parallel detection; in the controlled parallel detection, performing self-evaluation of brightness, contrast, sharpness, and noise on the adhesive coating area images of each effective camera, generating image quality self-evaluation results, and selectively performing denoising, sharpening, and brightness normalization processing based on the image quality self-evaluation results; loading an adhesive coating area ROI template; and performing template feature point matching within a preset matching band based on the ROI boundary to obtain a supplementary... The compensated ROI region is then segmented globally, with local threshold fine segmentation and morphological processing performed to obtain the adhesive path region contour. The adhesive path centerline is extracted from the contour, and the adhesive path boundary is searched along the centerline at equal intervals. The adhesive length, adhesive width, and width stability index are calculated, written into the region-level measurement completion marker, and the adhesive path geometric measurement data is output. Based on the adhesive path geometric measurement data, combined with the area difference, adhesive path centerline connectivity, adhesive path topological consistency, width stability index, and adhesive length range, a defect confidence vector is generated and weighted fusion is performed to obtain the fusion confidence. Based on the fusion confidence, the final defect judgment result of the adhesive coating region corresponding to each effective camera is determined, and the final defect judgment results of each effective camera under the same sequence number are temporally aligned and cross-view consistency verified, and the adhesive coating quality inspection result is output.

[0007] As a preferred embodiment of the visual inspection method for adhesive coating quality in a body-in-white welding line according to the present invention, the specific steps of establishing a PLC session and loading the desired camera set are as follows: Establish a detection session key using workstation ID and serial number, load the list of camera IDs for images to be acquired based on the white body model as the desired camera set, and initialize the already arrived camera set to an empty set; After receiving images of the coated area from multiple cameras, the corresponding camera ID is added to the set of arrived cameras, and a coverage judgment is made between the set of arrived cameras and the desired set of cameras.

[0008] As a preferred embodiment of the visual inspection method for adhesive coating quality in a body-in-white welding line according to the present invention, the specific steps of determining an effective camera and triggering controlled parallel detection are as follows: Cameras that have arrived in the camera set and belong to the desired camera set are identified as valid cameras; The number of parallel tasks is controlled based on the number of processor cores and the number of effective cameras of the industrial control computer, and independent detection tasks are assigned to each effective camera through semaphore throttling; After each independent detection task is completed, the corresponding camera ID, detection time, task status, and detection content are written into the shared result dictionary, and the contents of the shared result dictionary are stably sorted according to the camera ID.

[0009] As a preferred embodiment of the visual inspection method for adhesive coating quality in a body-in-white welding line according to the present invention, the specific steps for generating the image quality self-evaluation result are as follows: Four-dimensional indices of brightness, contrast, sharpness, and noise were calculated for the coated area images of each effective camera to obtain brightness, contrast, sharpness, and noise values. The brightness, contrast, and sharpness values ​​are positively weighted, while the noise value is negatively subtracted to obtain the overall quality score. The overall quality score is compared with the overall image quality assessment threshold, and the image quality self-assessment result is generated based on the comparison result; The overall image quality assessment threshold is determined during the initial debugging phase based on the total quality score of the defect-free standard samples.

[0010] As a preferred embodiment of the visual inspection method for adhesive coating quality in a body-in-white welding line according to the present invention, the specific steps of matching template feature points within a preset matching zone based on the ROI boundary are as follows: The ROI boundary is determined based on the ROI template of the adhesive application area. Using the ROI boundary as a reference, the ROI boundary is extended inward and outward to form a preset matching zone. The corner points and feature textures of the adhesive path edge are extracted within the preset matching band, and the corner points and feature textures are used as template feature points for matching, and the matching offset is calculated. The position of the ROI is corrected based on the matching offset to obtain the compensated ROI region.

[0011] As a preferred embodiment of the visual inspection method for adhesive coating quality in a body-in-white welding line according to the present invention, the specific steps of performing global coarse segmentation, local threshold fine segmentation, and morphological processing on the compensated ROI region are as follows: The adhesive path color threshold features in the RGB and HSV color spaces are extracted from the compensated ROI region, and global coarse segmentation is performed based on the adhesive path color threshold features to obtain candidate adhesive path regions. Within the candidate adhesive path region, local threshold fine segmentation is performed based on local window statistics, and the global coarse segmentation result and the local threshold fine segmentation result are fused to obtain the adhesive path segmentation mask; The adhesive path segmentation mask is expanded and etched to obtain the outline of the adhesive path region.

[0012] As a preferred embodiment of the visual inspection method for adhesive coating quality in a body-in-white welding line according to the present invention, the specific steps for calculating adhesive length, adhesive width, and width stability index are as follows: The center line of the adhesive path is obtained by extracting the skeleton of the adhesive path region contour, and the adhesive length is calculated based on the pixel length of the adhesive path center line and the pixel-to-millimeter conversion parameters. Sampling points are selected at equal intervals along the center line of the adhesive path, and the left and right boundaries of the adhesive path are searched along the normal direction of the center line of the adhesive path at each sampling point to calculate the adhesive width; The width stability index is calculated based on the variance of glue width at each sampling point and the change in glue width between adjacent sampling points.

[0013] As a preferred embodiment of the visual inspection method for adhesive coating quality in a body-in-white welding line according to the present invention, the specific steps for generating a defect confidence vector and performing weighted fusion to obtain a fused confidence score are as follows: The confidence level of glue leakage is determined based on the area difference and the connectivity of the glue path centerline; the confidence level of glue breakage is determined based on the topological consistency of the glue path; the confidence level of width fluctuation is determined based on the width stability; and the confidence level of glue length exceeding the limit is determined based on the glue length range. The defect confidence vector is composed of the confidence scores for glue leakage, glue breakage, width fluctuation, and glue length exceeding the limit. The defect confidence vector is then weighted according to the weights of each defect to obtain the fused confidence score. The fusion confidence level is compared with the comprehensive defect tolerance judgment threshold. When the fusion confidence level is lower than the comprehensive defect tolerance judgment threshold, the corresponding glue-coated area is determined to be unqualified. When the fusion confidence level is greater than or equal to the comprehensive defect tolerance judgment threshold, the corresponding glue-coated area is determined to be qualified. The weights for each defect are determined based on the severity of the impact of different defects on the final assembly safety of the vehicle body structure and the statistical results of on-site verification and debugging. The weight for glue breakage is the highest, followed by the weight for width fluctuation. The weights for the remaining defects are configured based on on-site verification data and process requirements.

[0014] As a preferred embodiment of the visual inspection method for adhesive coating quality in a body-in-white welding line according to the present invention, the specific steps for performing time-series alignment and cross-view consistency verification of the final defect judgment results of each valid camera under the same serial number are as follows: Based on the detection timestamp, the final defect determination results of each valid camera under the same serial number are included in the same time window; The glue path position, glue length, and glue width of the continuous glue application areas corresponding to adjacent cameras are compared. When at least one of the glue path position difference, glue length difference, and glue width difference exceeds the corresponding preset consistency range, the cross-view anomaly is marked as a consistency anomaly and incorporated into the glue application quality inspection result. The glue application quality inspection result is then output based on the final defect judgment result after verification.

[0015] Secondly, this invention provides a visual inspection system for adhesive coating quality in a body-in-white welding line, comprising: a session aggregation and parallel detection module, an image self-evaluation and ROI compensation module, an adhesive path segmentation and geometric measurement module, and a defect judgment and collaborative verification module; the session aggregation and parallel detection module is used to obtain the station ID, serial number, and body-in-white model after the body-in-white enters the welding inspection station, establish a PLC session and load the desired camera set, receive adhesive coating area images acquired by multiple cameras and update the arrived camera set, and determine the valid cameras and trigger controlled parallel detection when the arrived camera set covers the desired camera set and no timeout occurs; the image self-evaluation and ROI compensation module is used to perform brightness, contrast, sharpness, and noise self-evaluation on the adhesive coating area images of each valid camera in controlled parallel detection, generate image quality self-evaluation results, and selectively perform denoising, sharpening, and brightness normalization processing based on the image quality self-evaluation results, load the adhesive coating area ROI template, and calculate the ROI compensation based on the ROI. The I-boundary is used as a reference to perform template feature point matching within a preset matching band to obtain the compensated ROI region. The adhesive path segmentation and geometric measurement module is used to perform global coarse segmentation, local threshold fine segmentation, and morphological processing on the compensated ROI region to obtain the adhesive path region contour. The adhesive path centerline is extracted from the adhesive path region contour, and the adhesive path boundary is searched along the adhesive path centerline at equal intervals. The adhesive length, adhesive width, and width stability index are calculated, written into the region-level measurement completion mark, and the adhesive path geometric measurement data is output. The defect judgment and collaborative verification module is used to generate a defect confidence vector based on the adhesive path geometric measurement data, combined with the area difference, adhesive path centerline connectivity, adhesive path topology consistency, width stability index, and adhesive length range, and perform weighted fusion to obtain the fusion confidence. Based on the fusion confidence, the final defect judgment result of the adhesive area corresponding to each effective camera is determined, and the final defect judgment results of each effective camera under the same sequence number are temporally aligned and cross-view consistency verified, and the adhesive quality detection result is output.

[0016] The beneficial effects of this invention are as follows: It achieves convergence control for multi-camera acquisition and detection triggering through PLC sessions, desired camera sets, and arrived camera sets, reducing the problem of image-judgment mismatch; it improves the stability of glue path positioning and segmentation under complex lighting, reflection, shadow, and tooling deviations through image quality self-assessment, dynamic preprocessing, and ROI compensation; it reduces invalid measurement output by completing marking and glue path geometric measurement data through glue path centerline sampling and regional-level measurement; and it comprehensively judges defects such as glue leakage, glue breakage, width fluctuation, and glue length exceeding limits through defect confidence vectors, fusion confidence, temporal alignment, and cross-view consistency verification, improving the accuracy, stability, and traceability of glue coating quality detection results. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a visual inspection method for adhesive coating quality used in body-in-white welding lines.

[0019] Figure 2 This is a schematic diagram of a visual inspection system for adhesive coating quality used in a body-in-white welding line.

[0020] Figure 3 This is the overall flowchart of the visual inspection system for adhesive coating quality. Detailed Implementation

[0021] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0022] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0023] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0024] Reference Figures 1-3 As one embodiment of the present invention, this embodiment provides a visual inspection method for adhesive coating quality in a body-in-white welding line, comprising the following steps: S1. After the body-in-white enters the welding inspection station, the station ID, serial number and body-in-white model are obtained. A PLC session is established and the desired camera set is loaded. Images of the glue application area collected by multiple cameras are received and the set of cameras that have arrived is updated. When the set of cameras that have arrived covers the desired camera set and no timeout has occurred, the valid camera is determined and controlled parallel detection is triggered. Here, ID refers to: identifier; PLC refers to: programmable logic controller. It should be noted that this embodiment is applied to the glue coating quality inspection scenario of the body-in-white welding production line. The glue coating quality visual inspection system is installed next to the tooling at the body-in-white welding station to realize real-time detection of the glue coating quality of the body-in-white. The glue coating quality visual inspection system includes a multi-camera acquisition unit, an industrial control unit, an image transmission unit, a display and operation unit, and an installation and fixing unit. Each unit works together through a unified interface and event mechanism to adapt to the high-speed, multi-camera, and complex working condition inspection requirements of the body-in-white welding production line.

[0025] The unified interface includes a camera GigE interface and a PLC S7 / TCP communication interface; the event mechanism includes active polling events based on the PLC's operating status heartbeat cycle and image acquisition events based on trigger bit edge changes; the camera GigE interface is used to transmit images of the coating area acquired by the industrial area array camera, and the PLC S7 / TCP communication interface is used to realize the interaction of workstation signals, trigger signals, and detection result data between the PLC and the industrial control computer; the active polling event is used to periodically read the PLC's operating status, and the image acquisition event is used to start image acquisition from the corresponding camera when the PLC trigger bit changes; GigE refers to Gigabit Ethernet; Specifically, the multi-camera acquisition unit is equipped with an industrial area array camera and a telecentric lens according to the detection requirements of the glued area of ​​the body-in-white. The industrial area array camera has a resolution of no less than 2 million pixels and is fixed to the preset points of the welding fixture by an adjustable bracket to cover the glued areas such as welds and sealants on the body-in-white. The industrial area array camera is equipped with a supplementary lighting module, which uses a combination of ring light source and strip light source to improve the impact of uneven lighting on image acquisition at the welding site.

[0026] Preset points refer to the camera installation and shooting positions pre-configured in the visual inspection parameter template. The visual inspection parameter template is preset through the offline parameter calibration interface. During the preset process, the installation points of the corresponding cameras are determined according to the body-in-white model, the location of the adhesive application area, the camera's field of view, the telecentric lens working distance, and the direction of supplementary lighting, so that the cameras can cover the weld seams and sealant application areas corresponding to the current body-in-white model.

[0027] The industrial control unit includes an industrial-grade PLC and an industrial computer. The industrial-grade PLC, for example, is an S7-1500 PLC, used for communication with the main control system of the welding production line, acquisition of station signals, and writing back of detection results. The industrial computer uses a multi-core processor and has no less than 16G of memory, used for image processing, execution of detection algorithms, and parallel scheduling.

[0028] In this embodiment, the detection result refers to the specific Boolean judgment result including glue breakage, glue leakage, width abnormality, and glue length abnormality, as well as the corresponding measurement feature value; wherein, width abnormality includes glue piling up and thin glue, and glue length abnormality includes excessively long and excessively short; the measurement feature value includes glue path length, minimum glue width, maximum glue width, width variance, and maximum width gradient.

[0029] The image transmission unit uses gigabit industrial Ethernet to realize data transmission between the camera and the industrial computer, between the PLC and the industrial computer, and between the industrial computer and the main system of the production line, so as to ensure the real-time performance of image acquisition in the glue coating area, PLC session interaction and detection trigger command transmission.

[0030] The display and operation unit is used for setting detection parameters, configuring PLC parameters, displaying detection results, and tracing back images.

[0031] The mounting and fixing unit uses an adjustable metal bracket, and the camera bracket can be adjusted in the front-back, left-right, and up-down directions to adapt to the detection point adjustments of different body-in-white models.

[0032] Furthermore, the industrial control computer and PLC deploy software functional modules, including camera acquisition service, image hub, PLC session control module, parallel scheduling module, inspection service, image quality assessment and preprocessing module, ROI caching and dual-state display module, glue path geometry measurement module, defect judgment module, multi-camera collaborative verification module, online calibration module, image storage service, and log statistics module. The camera acquisition service controls multiple cameras to simultaneously acquire images of the glue-coated areas of the body-in-white and transmits the images to the image hub in real time. The image hub is used to achieve unified image reception, caching, and distribution, providing an image data source for subsequent processing. The PLC session control module communicates with the field PLC, establishes PLC sessions corresponding to workstations and serial numbers, manages the desired camera set and the set of arrived cameras, and controls the inspection triggering sequence. The parallel scheduling module is used to achieve controlled parallel inspection in debugging and mass production scenarios, dynamically adjusts the number of parallel tasks, and schedules the inspection service to perform parallel processing of multi-camera images. The system is designed for single-camera image detection tasks; the image quality assessment and preprocessing module performs four-dimensional quality scoring on acquired images and triggers adaptive preprocessing; the ROI caching and dual-state display module caches ROI contours and provides differentiated display for running and debugging states; the adhesive path geometry measurement module extracts the adhesive path centerline, calculates adhesive width and length, and adds region-level measurement markers; the defect judgment module combines adhesive path topology consistency, width stability, and area difference to perform multi-dimensional defect judgment and outputs the final defect judgment result through confidence fusion; the multi-camera collaborative verification module achieves temporal alignment and cross-view consistency verification of multi-camera detection results; the online calibration module periodically triggers calibration through a calibration board and dynamically updates pixel-to-millimeter conversion parameters; the image storage service stores detection images and results in separate directories for qualified and unqualified images; and the log statistics module records detection time, failure reasons, and detection process data. Here, Hub refers to the distribution center, and ROI refers to the point of interest. In this embodiment, the session aggregation and parallel detection module, image self-evaluation and ROI compensation module, adhesive path segmentation and geometric measurement module, and defect judgment and collaborative verification module are functional modules divided according to the processing stages of the adhesive coating quality visual inspection method. Specifically, the session aggregation and parallel detection module is implemented collaboratively by the camera acquisition service, image hub, PLC session control module, parallel scheduling module, and detection service; the image self-evaluation and ROI compensation module is implemented collaboratively by the image quality evaluation and preprocessing module, detection service, and ROI caching and dual-state display module; the adhesive path segmentation and geometric measurement module is implemented collaboratively by the detection service, adhesive path geometric measurement module, online calibration module, and ROI caching and dual-state display module; and the defect judgment and collaborative verification module is implemented collaboratively by the defect judgment module, multi-camera collaborative verification module, PLC session control module, image storage service, and log statistics module.

[0033] Specifically, after the body-in-white enters the welding and inspection station, the main system of the production line sends the station ID, serial number, and body-in-white model to the PLC. After receiving the station ID, serial number, and body-in-white model, the PLC session control module establishes a PLC session. The unique identifier of the PLC session is composed of the station ID (StationID) and the serial number (PartSerialNumber). The session object (StationSession) corresponding to the PLC session is bound to the currently assigned camera ID in memory through a dictionary. When the images from the specified camera are collected, the inspection results are written back to the PLC address dedicated to that station. Through the above binding method, a correspondence is formed between the current body-in-white, the current welding and inspection station, the current inspection task, the camera ID, the inspection image, the inspection result, and the PLC point, thereby ensuring that no data crosstalk occurs between different stations when multiple stations are inspected in parallel.

[0034] Furthermore, the PLC session control module loads a preset desired camera set based on the body-in-white model. The desired camera set is a list of camera IDs that need to acquire images for the current body-in-white model, used to characterize the range of cameras that should participate in the glue coating quality inspection for the current body-in-white model in the current welding inspection station. After loading the desired camera set, the PLC session control module initializes the arrived camera set and sets the arrived camera set to an empty set. The arrived camera set is used to record the camera IDs that have completed the glue coating area image acquisition and arrived at the image hub in the current PLC session.

[0035] Furthermore, the camera acquisition service controls multiple cameras to acquire images of the corresponding glue-coating area based on the body-in-white model and the desired camera set corresponding to the current PLC session. The glue-coating area images acquired by each camera are transmitted to the image hub through the image transmission unit. The image hub receives the glue-coating area images from each camera and synchronizes the image information to the PLC session control module. The image information includes at least the camera ID, acquisition timestamp, associated workstation ID, and trigger allocation sequence number. The PLC session control module determines the corresponding camera based on the camera ID in the image information and adds the corresponding camera ID to the set of arrived cameras, thus updating the set of arrived cameras.

[0036] The acquisition timestamp is used to compare with the trigger time of the current PLC session and participates in the image arrival timeout judgment and the timing alignment of subsequent multi-camera detection results; the associated station ID is used to verify whether the glue coating area image belongs to the current welding inspection station, so as to avoid images from different stations entering the same inspection session; the trigger-assigned sequence number is used to verify whether the glue coating area image belongs to the current body-in-white, so as to avoid the previous or next shot image being mixed with the current inspection task; the camera ID, acquisition timestamp, associated station ID and sequence number are synchronously written to the inspection log for subsequent image backtracking, failure cause location and inspection result traceability.

[0037] The PLC session control module compares the desired camera set with the arrived camera set in real time. When the arrived camera set completely covers the desired camera set and the preset timeout period has not been exceeded, the current PLC session is deemed to meet the detection trigger condition. The preset timeout period is set based on the PLC interaction timeout threshold and is determined in combination with the robot's motion stabilization time and network communication margin. For example, when the PLC cycle is usually around 1 second, the preset timeout period can be set to 500ms to balance image waiting time and error prevention requirements. If the image arrival delay exceeds the preset timeout period, the image is discarded to prevent the acquisition of an incorrect image in the next cycle. When the preset timeout period is exceeded, or when not all camera IDs in the desired camera set are added to the arrived camera set, the PLC session control module records the failure reason for the timeout or camera missing and writes the abnormal status back to the PLC.

[0038] After the current PLC session meets the detection trigger conditions, the PLC session control module triggers the detection command, and the parallel scheduling module starts the detection task. The parallel scheduling module determines the valid cameras based on the desired camera set and the already arrived camera set, and determines the cameras in the already arrived camera set that belong to the desired camera set as valid cameras. For camera IDs that have not captured images of the glued area, they are recorded as missing cameras and the corresponding failure reason is saved. Through the valid camera determination process, cameras with missing images or that do not belong to the detection range of the current body-in-white model can be excluded, avoiding invalid images from entering the subsequent detection process.

[0039] After identifying the valid cameras, the parallel scheduling module calculates the concurrency level based on the number of processor cores and the number of valid cameras in the industrial control computer. The concurrency level characterizes the number of independent detection tasks that can be executed in parallel at the same time, and is determined according to the following rule: Concurrency level = min(number of processor cores in the industrial control computer, number of valid cameras); where the number of processor cores in the industrial control computer is the number of processor cores that can be used for image processing and detection algorithm execution in the current industrial control computer, and the number of valid cameras is the number of valid cameras identified in the current PLC session. The parallel scheduling module sets the permitted number of semaphores according to the concurrency level, allocates independent detection tasks to the glued area images of each valid camera through semaphore throttling, and submits each independent detection task to the detection service for execution; when an independent detection task is completed and the semaphore is released, the parallel scheduling module continues to submit unexecuted valid camera detection tasks until all valid cameras in the current PLC session have completed detection.

[0040] Controlled parallel inspection is applicable to both mass production and debugging modes. In mass production mode, it is used for real-time inspection of the body-in-white after it enters the welding inspection station along the welding assembly line. In debugging mode, it is used for inspection and verification in scenarios such as parameter adjustment, problem tracing, and image group reuse. In mass production mode, the PLC session control module completes the coverage judgment of the expected camera set and the already arrived camera set according to the PLC session aggregation method, and triggers controlled parallel inspection after the detection triggering conditions are met. In debugging mode, controlled parallel inspection can also be combined with debugging image group cache reuse execution.

[0041] Each independent inspection task is executed independently within the inspection service, and the corresponding camera ID, inspection time, task status, and inspection content are written into the shared result dictionary. The shared result dictionary corresponds to the station inspection task session (StationSession) established in memory during the preceding photo triggering phase, used to record the inspection progress enumeration of the current station object. The inspection progress enumeration includes the states "triggered," "acquisition completed," and "inspection completed" in sequence, and is gradually updated as image quality self-assessment, ROI compensation, adhesive path segmentation, adhesive path geometric measurement, and defect judgment are executed. The inspection content includes the image quality self-assessment results, ROI compensation results, adhesive path geometric measurement data, final defect judgment results, and failure reasons. After each independent inspection task is completed, the parallel scheduling module stably sorts the contents of the shared result dictionary according to the camera ID. The stably sorted shared result dictionary is used for multi-camera collaborative verification, PLC result writing back, inspection log generation, and display interface status refresh, ensuring that the result output position of the same camera ID remains consistent in different inspection cycles. After the inspection task is completed, the PLC session control module releases the current PLC session resources and waits for the next white body station signal.

[0042] In debug mode, after the operator inputs the part serial number through the display and operation unit, the adhesive coating quality visual inspection system queries the image group cache using the model code, serial number, camera ID, and date as cache keys. If the cache is hit, the acquired image group is directly reused. If the cache is not hit, the system uses an intelligent image group retrieval method to search for images in the Teach, Meas, and Error directories according to a lenient serial number matching rule. The lenient serial number matching rule includes suffix matching and prefix extraction. When a direct and accurate match of the serial number cannot be achieved, the adhesive coating quality visual inspection system backtracks to historical groups and selects the image group containing the same suffix or the closest timestamp group. The close timestamp is usually limited to within 100ms before and after the current trigger signal. If multiple images meet the conditions, the system prioritizes deduplication by camera ID and selects the group of images whose timestamp is closest to the PLC trigger edge, assembling them into an image group and writing it into the cache. Here, Teach refers to teaching; Meas refers to measurement; and Error refers to error. Consistency checks are performed on image groups loaded in debug mode, retaining only images whose camera IDs belong to the expected camera set. For camera IDs missing from the expected camera set, a failure is recorded with the specific reason. After the image group consistency check is completed, the parallel scheduling module controls the number of parallel tasks based on the number of valid cameras, allocates independent detection tasks to the images of each valid camera through semaphore throttling, and submits them to the detection service. After each detection task is executed independently, the detection time, subsequent detection content, and task status are written to the shared result dictionary. After all tasks are completed, the data in the shared result dictionary is stably sorted according to the camera ID, and the overall detection time, detection time of each camera, and failure details are output for generating debug logs.

[0043] Stable sorting refers to arranging the detection data of each camera in the shared result dictionary into an array based on the physical hardware connection order of the cameras or the preset camera order in the configuration file, so that the output position of the results for the same camera ID remains consistent in different detection cycles.

[0044] S2. In controlled parallel detection, brightness, contrast, sharpness and noise are self-evaluated for the coated area images of each effective camera to generate image quality self-evaluation results. Based on the image quality self-evaluation results, denoising, sharpening and brightness normalization are selectively performed. The coated area ROI template is loaded, and template feature point matching is performed within the preset matching band with the ROI boundary as the reference to obtain the compensated ROI area.

[0045] It should be noted that after receiving the coated area images from each effective camera, the image quality assessment and preprocessing module performs four-dimensional index calculations on the coated area images, including brightness, contrast, sharpness, and noise, to obtain brightness, contrast, sharpness, and noise values, respectively. Among these, the brightness value is calculated using the image grayscale mean, the contrast value is calculated using the grayscale standard deviation, the sharpness value is calculated using the Laplacian variance, and the noise value is evaluated using high-frequency residuals. The above four-dimensional index calculations are implemented using conventional image quality assessment algorithms in this field.

[0046] The brightness, contrast, sharpness, and noise values ​​were normalized to convert the different dimensional indicators into dimensionless evaluation values, resulting in normalized brightness, contrast, sharpness, and noise values. The overall quality score was then calculated as follows: ; in, This indicates the overall quality score; This represents the normalized brightness value; This represents the normalized contrast value; This represents the normalized sharpness value; This represents the normalized noise value; , , and These represent the luminance weight, contrast weight, sharpness weight, and noise weight, respectively. , , and All are non-negative numbers, and .

[0047] Brightness, contrast, and sharpness values ​​are used as positive evaluation factors in the calculation of the overall quality score, while noise values ​​are used as negative evaluation factors. For example, , , and The value can be set to 0.25 for all positions; in workstations with unstable lighting, the brightness weight can be increased; when the imaging environment varies significantly between different camera positions, the values ​​can be recalibrated according to the corresponding camera positions. , , and .

[0048] Brightness, contrast, and sharpness values ​​are used as positive evaluation factors in the calculation of the overall quality score, while noise values ​​are used as negative evaluation factors.

[0049] Furthermore, the overall quality score will be... Image quality comprehensive evaluation threshold A comparison is performed, and the adaptive preprocessing link is triggered based on the comparison results; the overall image quality assessment threshold is calculated using the total quality score from defect-free standard samples during the initial debugging phase as the baseline score, and typically 80% of the baseline score is used as the alarm threshold; when the overall quality score... Less than the overall image quality assessment threshold When the corresponding glued area image is determined to be a low-quality image, the adaptive preprocessing link is triggered; when the overall quality score is... Greater than or equal to the overall image quality assessment threshold When the corresponding coated area image meets the subsequent detection requirements, the corresponding coated area image is allowed to enter the subsequent ROI drift compensation process.

[0050] The low-quality images that trigger the adaptive preprocessing link are sequentially processed by Gaussian denoising, Laplacian sharpening, and brightness normalization. Gaussian denoising is used to suppress industrial noise at the welding site of the body-in-white, Laplacian sharpening is used to improve the clarity of the glue path edges, and brightness normalization is used to solve the problems of uneven lighting and high reflectivity. Through denoising, sharpening, and brightness normalization, the low-quality glue-coated area images can obtain a more stable image foundation before entering ROI template matching and subsequent glue path segmentation.

[0051] The overall quality score was recalculated on the preprocessed image of the coated area. And the recalculated overall quality score Overall quality score before preprocessing Compare; when Greater than When, the pre-processed image of the coated area is selected for subsequent detection; when Less than or equal to When the image of the coated area before preprocessing is selected, it is used for subsequent detection. This image selection method avoids unnecessary impact on the original valid image when the preprocessing effect is poor.

[0052] After completing image quality self-assessment and dynamic preprocessing, the detection service loads a preset ROI template for the glued area based on the body-in-white model. The ROI template is entered in the parameter configuration interface of the background management system. The ROI template includes the ROI boundary, region ID, and detection indicators, which are used to determine the detection range and content of the glued area corresponding to each effective camera under the current body-in-white model. The detection indicators include the lower limit of area, the upper limit of area, the lower limit of width, the upper limit of width, the upper limit of continuous fine glue length, and the upper and lower limits of the overall glue length. The ROI boundary is used to limit the area range in the glued area image that needs to be detected for glue path detection, the region ID is used to identify the corresponding glued area, and the detection indicators are used to characterize the glue path segmentation, glue path geometric measurement, and defect judgment content that needs to be performed in this ROI area.

[0053] After loading the ROI template for the glue-coating area, a preset matching band is determined based on the ROI boundary. The matching band is the matching range of template feature points formed by extending inward and outward from the ROI boundary. The matching band corresponds to a small matching area formed by extending inward and outward from the ROI boundary. For example, the small matching area can be set to ±10 pixels. By limiting the matching of template feature points to the matching band, it is possible to avoid mismatches caused by unconstrained matching in the entire glue-coating area image, and at the same time, it can adapt to the ROI position shift caused by the slight deviation in the white body tooling positioning.

[0054] Furthermore, template feature points are extracted within the preset matching zone; the template feature points include corner points and feature textures of the glue path edge, and the corner points and feature textures are used to characterize the positions with stable visual features around the boundary of the glue application area ROI; specifically, the corner points and feature textures of the glue path edge are extracted within the preset matching zone through the contour search of the Halcon visual operator and the binarization morphology method, and the extracted corner points and feature textures are used as template feature points.

[0055] Furthermore, the detection service performs template matching using FindShapeModel to obtain the lateral translation X, vertical translation Y, and rotation angle. The lateral translation X represents the lateral offset of the actual adhesive path position in the current adhesive-coated area image relative to the ROI template; the vertical translation Y represents the vertical offset; and the rotation angle represents the angular offset. If multiple templates are used for matching, the lateral translation X, vertical translation Y, and rotation angle from the matching result with the highest confidence score are taken as the compensation offset for the overall ROI. Here, FindShapeModel refers to the shape feature operator, and Score refers to the fusion confidence score. Specifically, a local affine transformation matrix HomMat2D is established using the reference point found by template matching. The preset absolute ROI template is mapped to the actual pixel coordinate system corresponding to the current image through affine transformation, thereby completing the ROI position correction. After correction, the ROI boundary corresponding to the ROI template of the glued area is compensated and moved according to the actual glue path position in the current glued area image to obtain the compensated ROI area. The compensated ROI area is used to carry out subsequent global coarse segmentation, local threshold fine segmentation and morphological processing. HomMat2D refers to the two-dimensional homogeneous transformation matrix. S3. Perform global coarse segmentation, local threshold fine segmentation, and morphological processing on the compensated ROI region to obtain the contour of the adhesive path region. Extract the center line of the adhesive path from the contour of the adhesive path region, sample at equal intervals along the center line of the adhesive path and search for the boundary of the adhesive path, calculate the adhesive length, adhesive width, and width stability index, write the region-level measurement completion mark, and output the geometric measurement data of the adhesive path.

[0056] It should be noted that the inspection service performs adhesive path segmentation within the compensated ROI area, ensuring that the subsequent calculations of the adhesive path area contour, adhesive path centerline, adhesive length, and adhesive width are all based on the compensated ROI area, thereby avoiding inspection area offset due to minor tooling positioning deviations.

[0057] Specifically, global coarse segmentation is performed on the compensated ROI region; global coarse segmentation is used to obtain candidate adhesive path regions from the compensated ROI region; for dark adhesive paths, dark adhesive is extracted based on the threshold of each color channel in the RGB color space; for UV fluorescent adhesive paths, the adhesive path region is extracted based on the H and S components in the HSV color space; the adhesive path color threshold features include the minimum and maximum thresholds of each color channel, and global coarse segmentation is performed based on the adhesive path color threshold features to obtain candidate adhesive path regions, where RGB refers to: red, green, and blue; UV refers to: ultraviolet light; HSV refers to: hue, saturation, and brightness; Local threshold fine segmentation is performed within the candidate glue path region. This local threshold fine segmentation is based on an adaptive background mean separation threshold. Specifically, the original image is smoothed using a mean filter (MeanImage), and the smoothed image is subtracted from the scaled image to generate a dynamic reference surface, corresponding to the Halcon AutoThreshold operator, to form an adaptive background mean separation threshold. Based on this adaptive background mean separation threshold, the candidate glue path region obtained from the global coarse segmentation is further refined to reduce the impact of uneven lighting, high reflectivity, and shadow occlusion at the body-in-white welding site on glue path region recognition. The global coarse segmentation result is fused with the local threshold fine segmentation result to obtain a glue path segmentation mask, where MeanImage refers to the mean filter operator, and AutoThreshold refers to adaptive threshold segmentation. Morphological processing is performed on the adhesive path segmentation mask. This morphological processing uses a circular structuring element. Specifically, an opening operation with a radius of 1.0 is performed to remove burrs. After extracting the skeleton, a dilation operation is used to process the adhesive path endpoints and intersections, with an endpoint dilation radius of 10.5 and an intersection dilation radius of 5.5, used to break redundant connected regions at the intersections. The radius of the circular structuring element can be dynamically adjusted according to the adhesive path width. Through the above morphological processing, segmentation noise is eliminated, tiny holes in the adhesive path are filled, and a smooth adhesive path region contour is obtained.

[0058] The outline of the adhesive path region is extracted using a skeleton to obtain the centerline of the adhesive path. The centerline of the adhesive path is the central axis of the adhesive path region corresponding to the outline of the adhesive path region, and is used to characterize the extension direction of the adhesive path and the path for calculating the length of the adhesive path. Specifically, a distance map is generated using the DistanceTransform operator. After filtering the distance map using threshold segmentation, a thinning Skeleton operator is applied to extract the centerline of a single pixel width, resulting in a centerline point list. Here, DistanceTransform refers to distance transformation, and Skeleton refers to skeleton extraction. Calculate the glue length based on the glue path centerline; calculate and sum the distances between adjacent centerline points along the centerline point sequence to obtain the pixel length of the glue path centerline. ; via pixel-to-millimeter conversion parameters Convert the pixel length of the adhesive centerline to the actual physical length to obtain the adhesive length. The following formula can be used to calculate the glue length: ; ; in, Indicates the pixel length of the center line of the adhesive path; Indicates the actual physical length of the adhesive path; This represents the pixel-to-millimeter conversion parameter, with units of mm / px. , Indicates the first The pixel coordinates of the center line point in the image coordinate system; , Indicates the first The pixel coordinates of the center line point in the image coordinate system; This indicates the point number in the centerline point list of the adhesive path, and ; This indicates the number of centerline points in the centerline point column of the adhesive road.

[0059] Pixel to millimeter conversion parameters The image is captured by the online calibration module through a physical grid calibration board. Specifically, the online calibration module acquires images of the physical grid calibration board and uses the Halcon camera calibration algorithm to perform a geometric conversion between the pixel distance in the image and the physical millimeter size, obtaining the ratio coefficient between the pixel distance and the actual millimeter size, and using the ratio coefficient as the pixel-to-millimeter conversion parameter. The pixel-to-millimeter conversion parameter is used to ensure the accuracy of geometric measurements such as glue width and glue length.

[0060] The adhesive path width is calculated by sampling and searching the adhesive path boundary at equal intervals along the centerline. Specifically, M sampling points are sampled at equal intervals along the centerline of the adhesive path. M represents the number of sampling points used for width calculation and is determined in conjunction with the length of the adhesive path centerline and background noise filtering requirements. Line segments that are too short cannot calculate a stable normal angle. According to testing, skeleton lines with a pixel length less than 30 are usually considered background interference; therefore, M can be set between 20 and 30, and can be adjusted according to the adhesive path length. For each sampling point, a search line segment of a specified length is generated along the normal direction of the adhesive path centerline. The length of the line segment can be set to 1.5 times the maximum allowable adhesive width. The intersection points of the search line segment and the overall outer contour boundary of the adhesive path are calculated. These intersection points can be obtained using the IntersectionLineContourXld operator. IntersectionLineContourXld refers to the intersection point of the line and the XLD contour. The intersection points of the search line segment and the overall outer contour boundary of the adhesive path on both sides are taken as the left and right boundary points of the adhesive path. If no intersection point is found, it is considered that the adhesive path is severely missing at this location, and the local adhesive width of the corresponding sampling point is recorded as 0, thus obtaining the corresponding boundary point pair. and The local adhesive width is calculated based on the distance between boundary point pairs; the local adhesive width can be expressed by the following formula: ; in, Indicates sampling point The local glue width is wide; and These represent the sampling points. The left and right boundary points of the adhesive road are obtained by searching along the normal direction of the center line of the adhesive road; Indicates the sampling point index.

[0061] The width stability index is calculated based on the local adhesive width at each sampling point. The width stability index includes the width variance and the maximum width gradient. The width variance characterizes the overall fluctuation of the adhesive width along the centerline of the adhesive path, while the maximum width gradient characterizes the maximum amplitude of the adhesive width change between adjacent sampling points. Specifically, the local adhesive widths at each sampling point along the entire adhesive path skeleton are extracted. After calculating the average adhesive width, the discrete variance of each local adhesive width relative to the average adhesive width is calculated to obtain the width variance, expressed as: ; ; in, Indicates width variance; Indicates the average glue width; This indicates the total number of sampling points.

[0062] The maximum value among the changes in adhesive width between adjacent sampling points is calculated as the maximum width gradient, expressed as: ; in, This represents the maximum gradient across the width. Indicates the first The local glue width corresponding to each sampling point; This indicates the change in glue width between adjacent sampling points.

[0063] After calculating the glue length, glue width, and width stability indices, a region-level measurement completion marker is written into the detection results of this ROI region. This marker, called the region-level Measured marker, indicates whether the current ROI region has actually completed glue path geometric measurement. The condition for setting the region-level Measured marker to True is: the number of extracted connected component contours is greater than 0, and a valid length value and a valid minimum width value greater than 0 are successfully calculated. When these conditions are met, the region-level measurement completion marker is set to the measurement completion state. When the glue path region is not detected, or a valid length value and a valid minimum width value greater than 0 cannot be calculated, the region-level measurement completion marker is set to the unmeasured state. Here, Measured means: measurement completed; True means: enabled / true. When a region-level measurement is marked as completed, the adhesive path geometric measurement data for the current ROI region is output. This data includes adhesive length, adhesive width, and width stability indices, and is used in subsequent defect confidence calculations and final defect determination. When a region-level measurement is marked as incomplete, adhesive length, adhesive width, and width stability indices are not output, and this ROI region is not included in the defect confidence fusion calculation based on adhesive path geometric measurement data. Instead, this ROI region is marked as an incomplete measurement anomaly, and this anomaly is written as the cause of the detection failure in the final defect determination result. If an incomplete measurement anomaly exists in the same body-in-white, the adhesive coating quality inspection result is determined to be unqualified.

[0064] This embodiment also includes an ROI contour caching and dual-state display process; the ROI caching and dual-state display module caches ROI contours using model, camera, region ID, and ROI version number as cache keys; when the ROI parameters have not changed, the cached ROI contours are directly reused to avoid repeatedly constructing ROI contours and frequently redrawing images; in the running state, the display and operation unit only renders the ROI contours and defect marks of unqualified areas; in the debug state, the display and operation unit renders the ROI contours of all areas and overlays statistical text such as glue width and glue length.

[0065] S4. Based on the geometric measurement data of the adhesive path, combined with the area difference, the connectivity of the adhesive path centerline, the topological consistency of the adhesive path, the width stability index and the adhesive length range, a defect confidence vector is generated and weighted and fused to obtain the fusion confidence. Based on the fusion confidence, the final defect judgment result of the adhesive area corresponding to each effective camera is determined, and the final defect judgment results of each effective camera under the same serial number are time-series aligned and cross-view consistency verified, and the adhesive quality inspection result is output.

[0066] It should be noted that this step is executed after the glue path geometric measurement data is output, and is completed collaboratively by the defect judgment module and the multi-camera collaborative verification module. The defect judgment module, based on the glue path geometric measurement data, combines area difference, glue path centerline connectivity, glue path topology consistency, width stability index, and glue length range to perform multi-dimensional defect judgment and confidence fusion on the glue application area corresponding to each effective camera. The multi-camera collaborative verification module is used to perform time-series alignment and cross-view consistency verification of the final defect judgment results of each effective camera under the same serial number, and output the glue application quality inspection results.

[0067] When performing basic glue leakage judgment, the area difference between the ROI area and the actual glue path area is calculated. The ROI area is the area corresponding to the compensated ROI area, and the actual glue path area is the actual glue path area determined based on the glue path area contour. When the area difference is greater than the area difference threshold, it is determined that the current glue application area has a tendency to leak glue. When the glue path centerline connectivity test result is not connected, it is determined that the glue path centerline is broken, and the centerline break is used as the triggering basis for glue leakage judgment. The area difference threshold is determined by the historical statistical tolerance between the minimum envelope area and the maximum allowable glue overflow area of ​​the normal glue application sample, and is used to characterize the allowable glue path area missing range within the ROI area. When the area difference between the ROI area and the actual glue path area is greater than the area difference threshold, it is determined that the current glue application area has a tendency to leak glue.

[0068] When determining the connectivity of the glue path centerline, connectivity detection is performed on the centerline nodes to determine if there is a break in the glue path centerline. Continuous connection means that the width value of consecutive adjacent sampling points along the glue path centerline direction is not continuously lower than the set lower limit. Typically, the presence of a break in the glue path centerline is determined by whether the continuous thin glue running pixel length ThinRunLength is greater than a specified length threshold. If ThinRunLength is greater than the specified length threshold, the connectivity of the glue path centerline is determined to be abnormal, and the abnormal connectivity of the glue path centerline is used as the basis for determining glue leakage or glue breakage. Here, ThinRunLength refers to the continuous thin glue running pixel length. When determining the consistency of the adhesive path topology, the connectivity and self-intersection of the centerline nodes of the adhesive path are checked. If there is a break or self-intersection in the centerline of the adhesive path, it is determined to be a topology anomaly, and the topology anomaly is corresponding to the adhesive breakage defect.

[0069] When the width variance is greater than the width variance threshold, or the maximum width gradient is greater than the maximum width gradient threshold, it is determined that there is an abnormal width fluctuation in the current adhesive application area. The maximum width gradient threshold is calculated by the maximum adhesive path stacking slope allowed by the process, and the width variance threshold is determined according to the process requirement tolerance. For example, if the process requires the width fluctuation to be no more than 2mm, then 2mm is converted into the corresponding pixel size according to the pixel-to-millimeter conversion parameter and preset.

[0070] When determining if the glue length exceeds the limit, Compare with the upper and lower limits of the adhesive length required by the process, when If the glue length is lower than the lower limit required by the process or higher than the upper limit required by the process, it is determined that the glue length in the current glue application area exceeds the limit. The upper and lower limits of the glue length required by the process are determined by the standard physical length specified in the glue application process drawing of the white body at the workstation. A safety tolerance margin of 10% is added or subtracted from the standard physical length, and then converted into pixel value through pixel-to-millimeter conversion parameters for setting, which is used to limit the qualified glue length range of the current glue application area.

[0071] After obtaining the corresponding judgment criteria for glue leakage, glue breakage, width fluctuation, and glue length exceeding the limit, the defect judgment module generates a defect confidence vector. The defect confidence vector includes the confidence scores for glue leakage, glue breakage, width fluctuation, and glue length exceeding the limit. The defect confidence vector is represented as follows: ,in The confidence level for glue leakage is... For the confidence level of glue breakage, For the confidence level of width fluctuation, The confidence level for the gel length exceeding the limit is given.

[0072] In this embodiment, the defect confidence level is used to characterize the degree to which the corresponding detection index meets the process tolerance requirements, and does not represent the probability of defect occurrence. The closer the detection index is to the corresponding tolerance center value, the higher the defect confidence level. The greater the degree to which the detection index deviates from the corresponding tolerance center value or exceeds the process limit, the lower the defect confidence level.

[0073] Each defect confidence level is obtained by converting the corresponding judgment logic into a percentage score between 0 and 1. In this embodiment, the defect confidence level is used to characterize the degree to which the detection index meets the process tolerance requirements under the corresponding defect dimension. The closer the value is to 1, the closer the corresponding defect dimension is to the qualified state; the closer the value is to 0, the further the corresponding defect dimension deviates from the qualified state.

[0074] set up Indicates will Functions restricted to the range of 0 to 1, when When less than 0, take 0; when When it is greater than 1, take 1; when When it is between 0 and 1, take .

[0075] Confidence level of glue leakage Determine as follows: ; in, Indicates the confidence level of glue leakage; This represents the area difference between the ROI region area and the actual adhesive road area. Indicates the area difference threshold; This indicates the connectivity of the centerline of the adhesive path.

[0076] Confidence level of glue breakage Determine as follows: ; in, Indicates the confidence level of glue breakage; Indicates the pixel length of continuous fine adhesive application; Indicates a specified length threshold; This indicates a consistent topology for the adhesive path.

[0077] Width fluctuation confidence Determine as follows: ; in, Indicates the confidence level of width fluctuation; Indicates width variance; Indicates the width variance threshold; This represents the maximum gradient across the width. This represents the maximum gradient threshold.

[0078] Confidence level of gel length exceeding the limit Determine as follows: ; ; in, Indicates the confidence level of the gel length exceeding the limit; This indicates the deviation of the adhesive length from the acceptable range; This indicates the standard physical length specified in the white body adhesive application process drawings for the workstation. Indicates the actual physical length of the adhesive path; Indicates the lower limit of the adhesive length required by the process; This indicates the upper limit of the adhesive length required by the process.

[0079] When the same defect confidence level involves multiple judgment indicators, the minimum value is used as the corresponding defect confidence level so that the indicator with the most severe deviation can dominate the judgment result for this defect dimension; for example, when It is 20. When the value is 100 and the center line of the adhesive path is continuous, the confidence level of adhesive leakage is... It is 0.8; when the center line of the rubber track is discontinuous, Set the confidence level to 0 for glue leakage. The value is 0. In the above way, the area difference, connectivity status, topological consistency, width stability index and glue length range can be uniformly converted into a defect confidence value between 0 and 1.

[0080] The defect confidence vector is weighted and fused to obtain the fused confidence score. Specifically, the confidence scores in the defect confidence vector are weighted according to the weights corresponding to each defect to obtain the fused confidence score, expressed as follows: ; in, Indicates the fusion confidence level; Indicates the first Each defect corresponds to a weight; The first element in the defect confidence vector represents the... Confidence level of each defect; Indicates the defect type number.

[0081] The weight of each defect is determined by the severity of its impact on the final body structure assembly safety, and is obtained in combination with on-site verification and debugging statistics. Among them, glue breakage has the highest impact on the structural assembly safety and has the highest weight, followed by width fluctuation. The weights of the remaining defects are configured based on on-site verification data and process requirements.

[0082] The fusion confidence level is compared with the comprehensive defect tolerance threshold, and the final defect judgment result of the glued area corresponding to each effective camera is determined based on the fusion confidence level. The comprehensive defect tolerance threshold is used to characterize the minimum acceptable score after the weighted fusion of each defect confidence level. For example, the comprehensive defect tolerance threshold can be set to 0.8. If the fusion confidence level after the weighted fusion of each defect confidence level is lower than 0.8, the current glued area is judged to be unqualified. If the fusion confidence level is greater than or equal to 0.8, the current glued area is judged to be qualified.

[0083] The defect judgment module performs multi-dimensional defect judgment and confidence fusion processing on each glued area corresponding to each effective camera to obtain the final defect judgment result of the glued area corresponding to each effective camera. The final defect judgment result includes qualified judgment content and unqualified judgment content. When the judgment is unqualified, the final defect judgment result also includes the specific defect type, fusion confidence and evidence indicators, which are used for subsequent multi-camera collaborative verification, result writing back, image saving and log traceability.

[0084] The multi-camera collaborative verification module performs time-series alignment on the final defect judgment results of each valid camera under the same serial number. Specifically, the multi-camera collaborative verification module incorporates the final defect judgment results of each valid camera under the same serial number into the same time window based on the detection timestamp, thereby achieving time-series unification of multi-view detection results for the same body-in-white. The same time window can be set to ±100ms for example to avoid the detection results of different cameras not belonging to the same body-in-white detection cycle due to differences in acquisition or processing time.

[0085] The time window is set based on the cycle time of the body-in-white welding assembly line, the camera trigger cycle, and the average processing time of the inspection service. Under the condition that the results of multiple cameras can be aligned in time within the same body-in-white inspection cycle, it can be adjusted according to the cycle time of different production lines.

[0086] After timing alignment is completed, the multi-camera collaborative verification module performs cross-view consistency verification on the continuous adhesive application areas corresponding to adjacent cameras. Specifically, it compares the detection content of the continuous adhesive application area of ​​the body-in-white in the detection areas of adjacent cameras, including the adhesive path position, adhesive length, and adhesive width. When the difference in adhesive path position across views exceeds the preset consistency range, it is judged as a cross-view anomaly and added to the final defect judgment result. The consistency range can be set to ±5mm for example. Through cross-view consistency verification, contradictions in the determination of adhesive path position from different views can be avoided, and the reliability of multi-view detection results of the same body-in-white can be improved.

[0087] The consistency range is set based on the installation error, tooling positioning error, and pixel-to-millimeter conversion error of the overlapping area of ​​adjacent camera fields of view. Under the condition of meeting the consistency verification requirements of continuous adhesive road cross-view position, it can be adjusted according to different vehicle models and workstation configurations.

[0088] Both the same time window and the preset consistency range are empirical constants that balance detection accuracy and system performance, and can be obtained through training with multiple batches of samples. In one implementation, the same time window is set to ±100ms and the preset consistency range is set to ±5mm to achieve a balance between synchronizing multi-camera detection results and tolerating differences in cross-view detection. When the cycle time of different production lines, camera arrangement methods, or camera calibration errors change, the same time window and the preset consistency range can be adjusted according to the verification results of multiple batches of samples.

[0089] If a cross-view anomaly is detected, the PLC session control module sends a request to the PLC to re-capture the image, requiring the corresponding trigger camera to re-acquire the image of the corresponding continuous glue-coated area to verify the accuracy of the final defect judgment result. Re-triggered detection is usually limited to one retry. After re-detection, the verified final defect judgment result is included in the final defect judgment result of each valid camera under the same serial number, and timing alignment and cross-view consistency verification continue to be performed. If the two detection results are inconsistent, the last alarm result shall prevail to prevent missed detections.

[0090] After multi-camera collaborative verification is completed, the final defect judgment results of all valid cameras and their corresponding glue-coated areas are summarized to generate glue-coated quality inspection results. If any glue-coated area is judged as unqualified, the glue-coated quality inspection result of the current body-in-white is unqualified. If all glue-coated areas corresponding to all valid cameras are judged as qualified, the glue-coated quality inspection result of the current body-in-white is qualified. The glue-coated quality inspection result includes the qualified or unqualified judgment of the current body-in-white, the corresponding valid camera, the corresponding glue-coated area, the specific defect type, the fusion confidence level, and the evidence indicators.

[0091] The PLC session control module can write the glue application quality inspection results and specific failure reasons to preset PLC points. The main system of the production line reads the PLC point data in real time to achieve closed-loop control of the production line. The PLC points include qualified points and unqualified points. If the glue application quality inspection result of the current body-in-white is qualified, the qualified result is written to the qualified point. If the glue application quality inspection result of the current body-in-white is unqualified, the unqualified result and specific failure reasons are written to the unqualified point. The specific failure reasons include the valid camera number, glue application area number, and specific defect type.

[0092] The image storage service associates the detected images with the glue coating quality inspection results, storing the detected images and results in separate directories for qualified and unqualified items. The storage path includes the workstation ID, serial number, camera ID, and inspection time, and also saves the inspection result image with superimposed ROI contour, glue path geometric measurement data, and defect markers for subsequent traceability and analysis. The log statistics module records the inspection time, failure reasons, glue path geometric measurement data, final defect judgment results, and glue coating quality inspection results, enabling data statistics and traceability.

[0093] This embodiment can also be combined with the active learning of abnormal samples and the threshold self-adjustment process; specifically, the qualified or unqualified samples after manual review are statistically analyzed, and when the misjudged sample meets the threshold self-adjustment trigger condition, the interval convergence update of the area difference threshold, width variance threshold and width maximum gradient threshold is triggered.

[0094] The threshold self-adjustment trigger conditions include: within the preset statistical window, the number of manually reviewed samples reaches the sample number threshold, and the misjudgment rate of the corresponding defect type is greater than the misjudgment rate threshold; for example, the preset statistical window can be the most recent production shift or the most recent N consecutive white body pieces, the sample number threshold can be set to 20 pieces, and the misjudgment rate threshold can be set to 5%.

[0095] The interval convergence update uses a convergence update increment step size threshold to control the magnitude of a single update. An example value of the convergence update increment step size threshold is 0.05, which is used to control the iteration step size when updating the threshold and avoid oscillation caused by updating too quickly.

[0096] For the area difference threshold, width variance threshold, and width maximum gradient threshold, if the manual review results indicate that qualified samples are misjudged as unqualified, the corresponding thresholds are relaxed in the direction of reducing the misjudgment rate based on the corresponding feature statistics of qualified samples; if the manual review results indicate that unqualified samples are misjudged as qualified, the corresponding thresholds are tightened in the direction of reducing the false negative rate based on the corresponding feature statistics of unqualified samples. Each threshold adjustment does not exceed 1% of the current threshold, and is further verified through subsequent manual review samples after the update. Through threshold self-adjustment, the defect judgment algorithm can be optimized based on actual production line inspection data, achieving self-iterative optimization.

[0097] This embodiment also provides a visual inspection system for adhesive coating quality in a body-in-white welding line, including: a session aggregation and parallel detection module, an image self-evaluation and ROI compensation module, an adhesive path segmentation and geometric measurement module, and a defect judgment and collaborative verification module. The session aggregation and parallel detection module is used to obtain the station ID, serial number and body-in-white model after the body-in-white enters the welding inspection station, establish a PLC session and load the desired camera set, receive the glue application area images collected by multiple cameras and update the arrived camera set, and determine the valid camera and trigger controlled parallel detection when the arrived camera set covers the desired camera set and no timeout has occurred. The image self-evaluation and ROI compensation module is used to perform brightness, contrast, sharpness and noise self-evaluation on the coated area images of each effective camera in controlled parallel detection, generate image quality self-evaluation results, and selectively perform denoising, sharpening and brightness normalization processing based on the image quality self-evaluation results. It loads the coated area ROI template, and performs template feature point matching within a preset matching band based on the ROI boundary to obtain the compensated ROI area. The adhesive path segmentation and geometric measurement module is used to perform global coarse segmentation, local threshold fine segmentation and morphological processing on the compensated ROI region to obtain the outline of the adhesive path region. The center line of the adhesive path is extracted from the outline of the adhesive path region. The adhesive path boundary is searched by sampling at equal intervals along the center line of the adhesive path. The adhesive length, adhesive width and width stability index are calculated, written into the regional measurement completion mark and output the geometric measurement data of the adhesive path. The defect determination and collaborative verification module is used to generate a defect confidence vector based on the geometric measurement data of the adhesive path, combined with the area difference, the connectivity of the adhesive path centerline, the topological consistency of the adhesive path, the width stability index, and the adhesive length range. The vector is then weighted and fused to obtain the fusion confidence. Based on the fusion confidence, the final defect determination result of the adhesive area corresponding to each effective camera is determined. The final defect determination results of each effective camera under the same serial number are then time-aligned and cross-view consistency verified, and the adhesive quality inspection result is output.

[0098] In summary, this invention achieves convergence control for multi-camera acquisition and detection triggering through PLC sessions, desired camera sets, and arrived camera sets, reducing the problem of image-judgment mismatch; improves the stability of glue path positioning and segmentation under complex lighting, reflection, shadow, and tooling deviations through image quality self-assessment, dynamic preprocessing, and ROI compensation; reduces invalid measurement output by completing marking and glue path geometric measurement data through glue path centerline sampling and regional-level measurement; and improves the accuracy, stability, and traceability of glue coating quality detection results by comprehensively judging defects such as glue leakage, glue breakage, width fluctuation, and glue length exceeding limits through defect confidence vector, fusion confidence, temporal alignment, and cross-view consistency verification.

[0099] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A visual inspection method for adhesive coating quality in a body-in-white welding line, characterized in that: include: After the body-in-white enters the welding and inspection station, the station ID, serial number and body-in-white model are obtained, a PLC session is established and the desired camera set is loaded, the images of the glue application area collected by multiple cameras are received and the set of cameras that have arrived are updated, and when the set of cameras that have arrived covers the desired camera set and no timeout has occurred, the valid camera is determined and controlled parallel inspection is triggered. In controlled parallel detection, the brightness, contrast, sharpness and noise of the coated area images of each effective camera are self-evaluated to generate image quality self-evaluation results. Based on the image quality self-evaluation results, denoising, sharpening and brightness normalization are selectively performed. The coated area ROI template is loaded, and template feature point matching is performed within the preset matching band with the ROI boundary as the reference to obtain the compensated ROI area. Global coarse segmentation, local threshold fine segmentation and morphological processing are performed on the compensated ROI region to obtain the contour of the adhesive path region. The center line of the adhesive path is extracted from the contour of the adhesive path region. The adhesive path boundary is searched by sampling at equal intervals along the center line of the adhesive path. The adhesive length, adhesive width and width stability index are calculated, written into the regional measurement completion mark and the adhesive path geometric measurement data is output. Based on the geometric measurement data of the adhesive path, combined with the area difference, the connectivity of the adhesive path centerline, the topological consistency of the adhesive path, the width stability index, and the adhesive length range, a defect confidence vector is generated and weighted and fused to obtain the fusion confidence. Based on the fusion confidence, the final defect judgment result of the adhesive area corresponding to each effective camera is determined. The final defect judgment results of each effective camera under the same serial number are then time-aligned and cross-view consistency verified, and the adhesive quality inspection result is output.

2. The visual inspection method for adhesive coating quality in a body-in-white welding line as described in claim 1, characterized in that, The specific steps for establishing a PLC session and loading the desired camera set are as follows: Establish a detection session key using workstation ID and serial number, load the list of camera IDs for images to be acquired based on the white body model as the desired camera set, and initialize the already arrived camera set to an empty set; After receiving images of the coated area from multiple cameras, the corresponding camera ID is added to the set of arrived cameras, and a coverage judgment is made between the set of arrived cameras and the desired set of cameras.

3. The visual inspection method for adhesive coating quality in a body-in-white welding line as described in claim 2, characterized in that, The specific steps for determining a valid camera and triggering controlled parallel detection are as follows: Cameras that have arrived in the camera set and belong to the desired camera set are identified as valid cameras; The number of parallel tasks is controlled based on the number of processor cores and the number of effective cameras of the industrial control computer, and independent detection tasks are assigned to each effective camera through semaphore throttling; After each independent detection task is completed, the corresponding camera ID, detection time, task status, and detection content are written into the shared result dictionary, and the contents of the shared result dictionary are stably sorted according to the camera ID.

4. The visual inspection method for adhesive coating quality in a body-in-white welding line as described in claim 1, characterized in that, The specific steps for generating the image quality self-assessment result are as follows: Four-dimensional indices of brightness, contrast, sharpness, and noise were calculated for the coated area images of each effective camera to obtain brightness, contrast, sharpness, and noise values. The brightness, contrast, and sharpness values ​​are positively weighted, while the noise value is negatively subtracted to obtain the overall quality score. The overall quality score is compared with the overall image quality assessment threshold, and the image quality self-assessment result is generated based on the comparison result; The overall image quality assessment threshold is determined during the initial debugging phase based on the total quality score of the defect-free standard samples.

5. The visual inspection method for adhesive coating quality in a body-in-white welding line as described in claim 1, characterized in that, The specific steps for matching template feature points within a preset matching band based on the ROI boundary are as follows: The ROI boundary is determined based on the ROI template of the adhesive application area. Using the ROI boundary as a reference, the ROI boundary is extended inward and outward to form a preset matching zone. The corner points and feature textures of the adhesive path edge are extracted within the preset matching band, and the corner points and feature textures are used as template feature points for matching, and the matching offset is calculated. The position of the ROI is corrected based on the matching offset to obtain the compensated ROI region.

6. The visual inspection method for adhesive coating quality in a body-in-white welding line as described in claim 1, characterized in that, The steps for performing global coarse segmentation, local threshold fine segmentation, and morphological processing on the compensated ROI region are as follows: The adhesive path color threshold features in the RGB and HSV color spaces are extracted from the compensated ROI region, and global coarse segmentation is performed based on the adhesive path color threshold features to obtain candidate adhesive path regions. Within the candidate adhesive path region, local threshold fine segmentation is performed based on local window statistics, and the global coarse segmentation result and the local threshold fine segmentation result are fused to obtain the adhesive path segmentation mask; The adhesive path segmentation mask is expanded and etched to obtain the outline of the adhesive path region.

7. The visual inspection method for adhesive coating quality in a body-in-white welding line as described in claim 6, characterized in that, The specific steps for calculating the glue length, glue width, and width stability index are as follows: The center line of the adhesive path is obtained by extracting the skeleton of the adhesive path region contour, and the adhesive length is calculated based on the pixel length of the adhesive path center line and the pixel-to-millimeter conversion parameters. Sampling points are selected at equal intervals along the center line of the adhesive path, and the left and right boundaries of the adhesive path are searched along the normal direction of the center line of the adhesive path at each sampling point to calculate the adhesive width; The width stability index is calculated based on the variance of glue width at each sampling point and the change in glue width between adjacent sampling points.

8. The visual inspection method for adhesive coating quality in a body-in-white welding line as described in claim 1, characterized in that, The specific steps for generating the defect confidence vector and performing weighted fusion to obtain the fused confidence are as follows: The confidence level of glue leakage is determined based on the area difference and the connectivity of the glue path centerline; the confidence level of glue breakage is determined based on the topological consistency of the glue path; the confidence level of width fluctuation is determined based on the width stability; and the confidence level of glue length exceeding the limit is determined based on the glue length range. The defect confidence vector is composed of the confidence scores for glue leakage, glue breakage, width fluctuation, and glue length exceeding the limit. The defect confidence vector is then weighted according to the weights of each defect to obtain the fused confidence score. The fusion confidence level is compared with the comprehensive defect tolerance judgment threshold. When the fusion confidence level is lower than the comprehensive defect tolerance judgment threshold, the corresponding glue-coated area is determined to be unqualified. When the fusion confidence level is greater than or equal to the comprehensive defect tolerance judgment threshold, the corresponding glue-coated area is determined to be qualified. The weights for each defect are determined based on the severity of the impact of different defects on the final assembly safety of the vehicle body structure and the statistical results of on-site verification and debugging. The weight for glue breakage is the highest, followed by the weight for width fluctuation. The weights for the remaining defects are configured based on on-site verification data and process requirements.

9. A visual inspection method for adhesive coating quality in a body-in-white welding line as described in claim 8, characterized in that, The specific steps for performing time-series alignment and cross-view consistency verification on the final defect determination results of each valid camera under the same serial number are as follows: Based on the detection timestamp, the final defect determination results of each valid camera under the same serial number are included in the same time window; The glue path position, glue length, and glue width of the continuous glue application areas corresponding to adjacent cameras are compared. When at least one of the glue path position difference, glue length difference, and glue width difference exceeds the corresponding preset consistency range, the cross-view anomaly is marked as a consistency anomaly and incorporated into the glue application quality inspection result. The glue application quality inspection result is then output based on the final defect judgment result after verification.

10. A visual inspection system for adhesive coating quality in a body-in-white welding line, based on the visual inspection method for adhesive coating quality in a body-in-white welding line according to any one of claims 1 to 9, characterized in that, include: The module includes a session aggregation and parallel detection module, an image self-evaluation and ROI compensation module, an adhesive path segmentation and geometric measurement module, and a defect determination and collaborative verification module. The session aggregation and parallel detection module is used to obtain the station ID, serial number and body-in-white model after the body-in-white enters the welding inspection station, establish a PLC session and load the desired camera set, receive the glue application area images collected by multiple cameras and update the arrived camera set, and determine the valid camera and trigger controlled parallel detection when the arrived camera set covers the desired camera set and no timeout has occurred. The image self-evaluation and ROI compensation module is used to perform brightness, contrast, sharpness and noise self-evaluation on the coated area images of each effective camera in controlled parallel detection, generate image quality self-evaluation results, and selectively perform denoising, sharpening and brightness normalization processing based on the image quality self-evaluation results. It loads the coated area ROI template, and performs template feature point matching within a preset matching band based on the ROI boundary to obtain the compensated ROI area. The adhesive path segmentation and geometric measurement module is used to perform global coarse segmentation, local threshold fine segmentation and morphological processing on the compensated ROI region to obtain the outline of the adhesive path region. The center line of the adhesive path is extracted from the outline of the adhesive path region. The adhesive path boundary is searched by sampling at equal intervals along the center line of the adhesive path. The adhesive length, adhesive width and width stability index are calculated, written into the regional measurement completion mark and output the geometric measurement data of the adhesive path. The defect determination and collaborative verification module is used to generate a defect confidence vector based on the geometric measurement data of the adhesive path, combined with the area difference, the connectivity of the adhesive path centerline, the topological consistency of the adhesive path, the width stability index, and the adhesive length range. The vector is then weighted and fused to obtain the fusion confidence. Based on the fusion confidence, the final defect determination result of the adhesive area corresponding to each effective camera is determined. The final defect determination results of each effective camera under the same serial number are then time-aligned and cross-view consistency verified, and the adhesive quality inspection result is output.

Images