ECU shell cover sealing detection system based on image recognition

Abstract

The present application relates to the field of machine vision and automatic detection technology, specifically to an ECU shell cover sealing detection system based on image recognition, comprising an industrial camera, a light source assembly, a rejection mechanism and a data processing terminal; the data processing terminal runs image acquisition, image alignment, highlight extraction, displacement calculation, data checking, trajectory analysis, continuity judgment and re-inspection control modules; by continuously collecting the same ECU shell cover image sequence, extracting the positioning reference and aligning, matching the standard sealing glue trajectory, extracting the highlight pixel area in the strip area and morphologically expanding to obtain the detection area, calculating the adjacent frame pixel displacement vector and the overall displacement, obtaining the effective displacement vector after eliminating the influence of the overall movement of the workpiece, analyzing the segmented detection window according to the data validity mark, judging the continuity of the sealing glue according to the adjacent window detection feature data and the displacement direction and amplitude change, and executing rejection or re-shooting re-inspection according to the result.

Description

Technical Field

[0001] This invention relates to the field of machine vision and automated inspection technology, specifically to an ECU housing cover sealing inspection system based on image recognition. Background Technology

[0002] During the production of automotive electronic control unit housing covers, the continuity of the housing cover sealant directly affects the product's waterproof and dustproof performance. Therefore, it is usually necessary to quickly test the housing cover's sealing condition after assembly in order to promptly detect abnormalities such as broken sealant or insufficient sealant. Existing detection methods mostly rely on manual visual inspection or simple recognition methods based on single-frame images. These methods are easily affected by factors such as workpiece displacement, positioning deviation, reflection interference, and unstable acquisition timing, resulting in fluctuations in detection results exceeding the preset threshold. It is difficult to distinguish between glue breakage, abnormal reflection, and image failure, thus affecting detection accuracy and online rejection efficiency. Summary of the Invention

[0003] The main objective of this invention is to propose an ECU housing cover sealing detection system based on image recognition. This system combines continuous image sequence acquisition and alignment, highlight trajectory analysis to extract effective displacement vectors, and data verification and re-inspection control. This enables the system to effectively overcome the influence of factors such as workpiece displacement, positioning deviation, reflective interference, and unstable acquisition timing when detecting the sealing status of the housing cover. It can accurately distinguish between genuine glue breakage and abnormal reflection or image failure, thereby improving detection accuracy and online rejection efficiency.

[0004] The objective of this invention can be achieved through the following technical solutions: An image recognition-based ECU housing cover sealing inspection system includes an industrial camera, a light source assembly, a rejection mechanism, and a data processing terminal communicatively connected to the industrial camera and the rejection mechanism. The data processing terminal operates as follows: The image acquisition module controls the industrial camera to continuously acquire a preset number of image sequences of the same ECU housing cover when the light source component is turned on; The image alignment module extracts the positioning reference based on the template data, aligns each frame of image to the same coordinate system, outputs the aligned image and alignment error, and matches the standard sealant trajectory. The highlight extraction module extends a preset pixel distance along both sides of the trajectory to form a strip-shaped area. The highlighted pixel area extracted within the strip-shaped area is morphologically expanded to obtain the detection area, and the effective detection area percentage is output. The displacement calculation module calculates the displacement vector of adjacent frame pixels and the overall displacement of the ECU cover within the detection area. After coordinate transformation, the overall displacement of the ECU cover is eliminated to obtain the effective displacement vector. The data verification module verifies the alignment error, overall displacement, reliability of the effective displacement vector, and proportion of the effective detection area, and outputs data validity labels. The trajectory analysis module divides the trajectory into segmented detection windows and calculates the detection feature data within each window based on the effective displacement vector with data validity markers. The continuity judgment module determines the continuity of the sealant and outputs the detection results based on the detection feature data of adjacent windows and the changes in displacement direction and amplitude between windows; The re-inspection control module, based on the test results, controls the rejection mechanism to reject the product when it is broken, and controls the industrial camera to take a new picture when there is an abnormality or invalid data.

[0005] Preferably, the system also includes a position sensor that is connected to the data processing terminal. The image acquisition module continuously acquires image sequences and records the acquisition time interval within a preset time after receiving the ECU cover positioning signal from the position sensor. The displacement calculation module calculates the overall displacement of the ECU housing between two adjacent frames and determines whether the overall displacement is within a preset effective range. When the overall displacement is less than the first displacement threshold or greater than the second displacement threshold, the current image sequence is determined to be invalid and a delayed reshoot is triggered. When the overall displacement is within the preset effective range, the image sequence is sent to the highlight extraction module.

[0006] Preferably, the image alignment module identifies at least one of the positioning holes, corner points, or outlines used as positioning references, calculates the transformation matrix required for image alignment of each frame, and obtains the alignment error by evaluating the image alignment quality based on the alignment error of the positioning references. When the alignment error of any frame exceeds the preset alignment error threshold, repositioning or reshooting is triggered, the image sequence is marked as invalid and prohibited from entering the continuity judgment process; when the alignment error of any frame is less than or equal to the preset alignment error threshold, the aligned image is output.

[0007] As a preferred embodiment, the highlight extraction module performs binarization processing on the aligned image to extract the highlight pixel region, and filters out abnormal reflective areas within the strip region according to a preset area threshold. The highlight extraction module expands the filtered highlight pixel area outward by a preset number of pixels and intersects it with the strip area to obtain the detection area; When the number of highlight pixels in the strip area is less than the preset highlight pixel number threshold, an invalid detection status is output, and the industrial camera exposure adjustment or re-shooting and re-inspection is triggered.

[0008] Preferably, the displacement calculation module calculates the forward pixel displacement and the reverse pixel displacement in the detection area of ​​two adjacent frames of images, and retains the vectors that meet the conditions according to the preset error threshold, the preset grayscale change threshold and the preset displacement range threshold. The displacement calculation module corrects the positive pixel displacement based on the transformation matrix of the image alignment module to obtain an effective displacement vector that eliminates the overall displacement of the ECU cover.

[0009] Preferably, the detection feature data includes the degree of consistency between the direction of the effective displacement vector and the trajectory tangent direction, the coverage of the effective displacement vector, the amount of local pixel change, the sustained occurrence rate of highlights, and the overlap rate of highlights in adjacent frames. The trajectory analysis module calculates the detection feature data within each segment detection window; among them, the local pixel change is calculated only using the effective displacement vectors whose corresponding pixel's preset neighborhood window is located within the detection area. When the number of effective displacement vectors is lower than the preset threshold, the corresponding local pixel change is marked as invalid. When the coverage of the effective displacement vector is lower than the preset coverage threshold, the corresponding directional consistency data is not included in the judgment of sealant continuity.

[0010] Preferably, the trajectory analysis module predicts the position of the current frame's highlight pixel region in the next frame based on the effective displacement vector, and calculates the overlap area ratio between the predicted position and the highlight pixel region in the next frame to obtain the highlight overlap rate between adjacent frames. The sustained occurrence rate of highlights is determined by statistically analyzing the proportion of frames in each segment detection window where the number of highlight pixels is greater than or equal to a preset threshold for the number of highlight pixels to the total number of frames. The trajectory analysis module determines the reflection type label based on whether the sustained occurrence rate of highlights and the overlap rate of highlights in adjacent frames meet preset thresholds.

[0011] As a preferred embodiment, the continuity judgment module uses each segment detection window as a judgment node. When the coverage of the effective displacement vector and the sustained occurrence rate of the highlight meet the preset confidence thresholds, the corresponding judgment node is marked as a confidence node. For adjacent reliable nodes that meet the preset threshold conditions of direction consistency, local pixel change, specular persistence rate and adjacent frame specular overlap rate, a connection relationship is established based on the angle and magnitude difference of the direction change of the average displacement vector between the two nodes. Windows that have not established connections or do not meet the trust threshold are marked as continuous abnormal windows, and the connectivity of the sealant is calculated based on the proportion of node connections and the penalty coefficient of the length of continuous abnormal windows. For the standard sealant trajectory in a closed loop, the continuity judgment module also determines the connection relationship between the start and end point segment detection windows.

[0012] As a preferred option, the re-inspection control module determines the test results as sealant continuity, suspected sealant breakage, confirmed sealant breakage, or invalid data based on data validity marking, the proportion of reliable nodes, the length of continuous abnormal windows, and the consistency of results before and after re-shooting. Among them, invalid data status triggers ECU cover repositioning, industrial camera exposure adjustment, or output of manual re-inspection prompts; suspected glue breakage status triggers in-situ reshooting. If the length of consecutive abnormal windows within the same trajectory segment exceeds a preset length threshold, and consecutive abnormal windows still appear in the same trajectory segment after reshooting, output confirmation of glue breakage status.

[0013] As a preferred option, the data processing terminal also runs a parameter setting module. The parameter setting module generates detection parameters based on the results of manual review or standard sample testing, and when the detection misjudgment rate exceeds the preset misjudgment rate threshold within the preset statistical period, the detection parameters are rolled back to the previous parameter version.

[0014] The beneficial effects of this invention are: 1. This invention can effectively eliminate the influence of workpiece displacement and positioning deviation on the detection results by continuously acquiring a preset number of images of the same workpiece and combining them with template data for alignment processing, thus avoiding misjudgment caused by posture changes in single-frame recognition; by setting a strip area along the standard sealant trajectory and extracting the highlight pixel area, the analysis range can be limited to the vicinity of the sealant, reducing the influence of reflection interference outside the trajectory on the judgment and improving the stability of highlight evidence. 2. This invention triggers re-shooting when data is invalid or abnormal, and controls the rejection mechanism to perform rejection action when glue breakage is confirmed. This enables closed-loop linkage of detection, re-inspection and rejection, improving the reliability and automation level of the entire line detection. The parameter setting module can also roll back the detection parameters and quickly restore the system to the verified parameters when the false judgment rate increases, further ensuring the long-term stability of the system. Attached Figure Description

[0015] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a block diagram of an ECU housing cover sealing detection system based on image recognition, according to an embodiment of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Please see Figure 1 An image recognition-based ECU housing cover sealing inspection system includes an industrial camera, a light source assembly, a rejection mechanism, and a data processing terminal that communicates with the industrial camera and the rejection mechanism. The data processing terminal operates as follows: The image acquisition module controls the industrial camera to continuously acquire a preset number of image sequences of the same ECU housing cover when the light source component is turned on; The image alignment module extracts the positioning reference based on the template data, aligns each frame of image to the same coordinate system, outputs the aligned image and alignment error, and matches the standard sealant trajectory. The highlight extraction module extends a preset pixel distance along both sides of the trajectory to form a strip-shaped area. The highlighted pixel area extracted within the strip-shaped area is morphologically expanded to obtain the detection area, and the effective detection area percentage is output. The displacement calculation module calculates the displacement vector of adjacent frame pixels and the overall displacement of the ECU cover within the detection area. After coordinate transformation, the overall displacement of the ECU cover is eliminated to obtain the effective displacement vector. The data verification module verifies the alignment error, overall displacement, reliability of the effective displacement vector, and proportion of the effective detection area, and outputs data validity labels. The trajectory analysis module divides the trajectory into segmented detection windows and calculates the detection feature data within each window based on the effective displacement vector with data validity markers. The continuity judgment module determines the continuity of the sealant and outputs the detection results based on the detection feature data of adjacent windows and the changes in displacement direction and amplitude between windows; The re-inspection control module, based on the test results, controls the rejection mechanism to reject the product when it is broken, and controls the industrial camera to take a new picture when there is an abnormality or invalid data.

[0018] In this embodiment, an industrial camera, a light source assembly, a rejection mechanism, and a data processing terminal constitute a detection execution unit. After the light source assembly is turned on, the industrial camera acquires continuous images of the same ECU housing. The image acquisition module outputs an image sequence with a preset number of frames according to the system cycle. The preset number of frames is used to ensure that the same workpiece is sampled multiple times in a short-term continuous state, thereby obtaining the inter-frame differences required for subsequent displacement analysis. The image alignment module first calls the pre-acquired ECU cover template data and extracts the positioning reference from the image. The positioning reference can come from at least one of the positioning holes, corner points, or outer contours. The module calculates the transformation relationship of each frame of image based on the positioning reference, unifies each frame of image into the same coordinate system, and outputs the aligned image and alignment error; Alignment error is used to reflect the magnitude of the deviation between each frame image and the template. When the deviation meets the preset rules, the aligned image will proceed to subsequent processing; when the deviation does not meet the preset rules, the image set will not participate in the sealing judgment. After alignment is completed, the data processing terminal calls the highlight extraction module; this module extends outward by a preset pixel distance along both sides of the standard sealant track to form a strip area; the function of the strip area is to limit the analysis range of the sealant and avoid including highlight interference outside the track in the calculation. The module extracts the highlight pixel regions within the strip-shaped area, then performs morphological expansion on the highlight pixel regions to obtain the detection region, and outputs the effective detection region ratio; the effective detection region ratio is used to reflect whether there is enough region in the current frame that can be used for subsequent displacement judgment. The displacement calculation module calculates the pixel displacement vector between two adjacent frames within the detection area. At the same time, it combines the transformation relationship output by the alignment module to calculate the overall displacement of the ECU cover. Then, it eliminates the overall displacement from the pixel displacement vector through coordinate transformation to obtain the effective displacement vector. After eliminating the displacement component caused by the workpiece's own movement through coordinate transformation, the subsequent judgment is based on the change of the sealant highlight pixel area relative to the workpiece coordinate system. The data verification module verifies the alignment error, overall displacement, reliability of the effective displacement vector, and proportion of the effective detection area; if any of these items does not meet the preset rules, the output data validity is marked as invalid. If the preset rules are met, the output data is marked as valid; this mark is directly used as the input condition for the trajectory analysis module, and only valid data will enter the segment detection window for analysis. The trajectory analysis module divides the standard sealant trajectory into multiple segmented detection windows, and calculates the detection feature data within each segmented detection window based on the valid displacement vector with data validity markers. The continuity judgment module then judges the continuity of the sealant based on the detection feature data of adjacent segment detection windows, as well as the changes in displacement direction and amplitude between adjacent windows, and outputs detection results such as continuous, suspected sealant breakage, sealant breakage or invalid data. The re-inspection control module performs subsequent actions based on the result: if the glue is broken, it controls the rejection mechanism to reject the product; if the result is abnormal or the data is invalid, it controls the industrial camera to take a retake shot to reacquire the image sequence and re-enter the judgment process. The image acquisition module uses a single-workpiece corresponding cyclic buffer method when performing continuous acquisition. Each frame in the buffer is appended with an acquisition time stamp, alignment status and integrity status before entering the subsequent module. The integrity status is used to characterize whether the current frame has exposure saturation, abnormal inter-frame interval or missing frames in the buffer. If the current frame has any of the above situations that cause the integrity status to be abnormal, it will be directly marked as an invalid frame and removed from the subsequent calculation link. When the preset number of frames has not been completed in continuous acquisition, the frames that have passed the verification in the previous sequence remain in the pending judgment state and no control command is output to the rejection mechanism. The image alignment module, highlight extraction module, displacement calculation module, and trajectory analysis module are sequentially linked in a frame-level order. When any module outputs an invalid mark, the subsequent modules only receive the invalid mark and stop numerical calculation. The analysis link for the same workpiece is restarted only after the re-inspection control module completes the re-shooting, thus forming a closed-loop topology driven by the validity mark. In the case of multiple workpieces entering continuously, the data processing terminal maintains parallel but non-overlapping buffer segments according to the workpiece number. Each buffer segment is released immediately after the corresponding workpiece is judged or is determined to be invalid, so as to avoid cross-contamination of frame sequences of different workpieces.

[0019] In one alternative, the system also includes a position sensor that is communicatively connected to the data processing terminal. The image acquisition module continuously acquires image sequences and records the acquisition time interval within a preset time after receiving the ECU cover positioning signal from the position sensor. The displacement calculation module calculates the overall displacement of the ECU housing between two adjacent frames and determines whether the overall displacement is within a preset effective range. When the overall displacement is less than the first displacement threshold or greater than the second displacement threshold, the current image sequence is determined to be invalid and a delayed reshoot is triggered. When the overall displacement is within the preset effective range, the image sequence is sent to the highlight extraction module.

[0020] In this embodiment, the position sensor is used to confirm whether the ECU cover has entered the collectable area; after the position sensor sends a position signal to the data processing terminal, the image acquisition module starts continuous acquisition within a preset time and records the acquisition time interval between each frame. By making the acquisition action correspond to the workpiece's position, we can prevent invalid data from being acquired when the workpiece is not in position. The displacement calculation module calculates the overall displacement of the ECU housing between two adjacent image frames; this overall displacement reflects the degree of position change of the workpiece during the acquisition process, and the module compares this displacement with a preset effective range; When the overall displacement is less than the first displacement threshold, it indicates that the workpiece displacement has not reached the preset lower displacement limit and the highlight change in the image sequence is less than the preset change threshold; when the overall displacement is greater than the second displacement threshold, it indicates that the workpiece displacement is greater than the preset upper displacement limit and the rate of change between images is greater than the preset rate of change threshold, and the valid judgment condition is not met. In both cases, the current image sequence is determined to be invalid, and a delayed reshoot is triggered; if the overall displacement is within the preset valid range, the image sequence is sent to the highlight extraction module and displacement calculation module for further processing. The overall displacement is statistically calculated using the displacement unit consistent with the image pixel coordinates, and the translation component amplitude obtained before and after alignment of the positioning reference in two adjacent frames is used as the calculation basis. When the acquisition time interval recorded by the image acquisition module is missing, abnormally jumps, or exceeds the preset tolerance, the current image sequence is directly determined to be invalid, so as to avoid comparing the displacement under different time spans when the time base is inconsistent. The preset effective range is compared with the first displacement threshold and the second displacement threshold in the same displacement unit, and the first displacement threshold is less than the second displacement threshold, so as to ensure that the lower limit judgment and the upper limit judgment are performed under the same dimension, and avoid threshold crossing or judgment interval inversion. The preset time, first displacement threshold, second displacement threshold, and preset effective range are all obtained from the statistical results of the standard sample under different conveying speeds and attitude deviations. The range in which the pass rate of the effective sample meets the preset requirements and the misjudgment rate is less than the preset misjudgment threshold is taken as the detection parameter. The delay time of the delayed supplementary shooting is consistent with the acquisition time interval.

[0021] In one alternative, the image alignment module calculates the transformation matrix required for image alignment of each frame by identifying at least one of the positioning holes, corner points, or outlines used as positioning references, and obtains the alignment error by evaluating the image alignment quality based on the alignment error of the positioning references. When the alignment error of any frame exceeds the preset alignment error threshold, repositioning or reshooting is triggered, the image sequence is marked as invalid and prohibited from entering the continuity judgment process; when the alignment error of any frame is less than or equal to the preset alignment error threshold, the aligned image is output.

[0022] In this embodiment, the image alignment module identifies at least one of the following as positioning references: positioning holes, corner points, or outlines. After identifying the positioning reference, the module calculates the transformation matrix required for image alignment of each frame and completes the image alignment accordingly. The alignment error is obtained by calculating the root mean square distance of the feature points of the positioning reference before and after alignment, using the following formula: in, For alignment error, The number of positioning reference points, This is the location reference point number. and The first and second positions before and after the alignment The error is used to reflect the degree to which the current image fits the unified coordinate system, and the reference point pixel coordinates are used to reflect the degree to which the current image fits the unified coordinate system. When the alignment error of any frame exceeds the preset alignment error threshold, the image alignment module marks the image sequence as invalid and outputs a repositioning or reshooting instruction to the data processing terminal, so that the group of images does not enter the continuity judgment process. To prevent trajectory misalignment caused by positioning deviation from affecting the seal connectivity determination; if the alignment error of any frame does not exceed the preset alignment error threshold, the image alignment module outputs the aligned image for subsequent highlight extraction and displacement calculation.

[0023] In one alternative, the highlight extraction module performs binarization processing on the aligned image to extract the highlight pixel region, and filters out abnormal reflective areas within the strip region according to a preset area threshold. The highlight extraction module expands the filtered highlight pixel area outward by a preset number of pixels and intersects it with the strip area to obtain the detection area; When the number of highlight pixels in the strip area is less than the preset highlight pixel number threshold, an invalid detection status is output, and the industrial camera exposure adjustment or re-shooting and re-inspection is triggered.

[0024] In this embodiment, after receiving the aligned image, the highlight extraction module first performs binarization processing on the image and extracts the pixels that meet the highlight conditions as highlight pixels. The module performs area screening on the connected regions formed by the highlight pixels within the strip area, and uses a preset area threshold to filter out abnormal reflective areas to prevent non-target reflections with an area larger than the preset reflective area threshold from affecting the judgment. The filtered highlight pixel area is then reused with a size of All elements are The square structural element, in which To predetermine the number of dilated pixels, a morphological dilation operation is performed, and the intersection with the strip region is taken to obtain the detection area. This detection area is used to limit the scope of subsequent displacement calculations, so that the analysis focuses on the highlight changes near the standard sealant trajectory. When the number of highlight pixels in the strip area is less than the preset highlight pixel count threshold, it indicates that there is insufficient highlight evidence in the current image for judgment. The highlight extraction module outputs an invalid detection status and triggers the industrial camera to adjust the exposure or retake the image for re-inspection in order to obtain an image sequence that meets the requirements. The preset area threshold and the preset highlight pixel number threshold are determined by the statistical analysis of abnormal reflections in the standard sample image, and the upper limit values ​​that do not affect the continuity of the highlight of the normal sealant are taken respectively; the exposure adjustment of the industrial camera is performed by increasing or decreasing by a preset exposure step size in adjacent levels, and the preset exposure step size is the minimum difference between adjacent exposure levels that the camera can set.

[0025] In one alternative, the displacement calculation module calculates the forward pixel displacement and the reverse pixel displacement in the detection area of ​​two adjacent frames of images, and retains the vectors that meet the conditions according to the preset error threshold, the preset grayscale change threshold and the preset displacement range threshold. The displacement calculation module corrects the positive pixel displacement based on the transformation matrix of the image alignment module to obtain an effective displacement vector that eliminates the overall displacement of the ECU cover.

[0026] In this embodiment, the displacement calculation module only calculates the pixel displacement between two adjacent frames within the detection area; it calculates the forward pixel displacement, that is, the displacement from the previous frame to the next frame; and then calculates the reverse pixel displacement, that is, the displacement from the next frame back to the previous frame. The module cross-checks the forward pixel displacement and the reverse pixel displacement, and combines the preset error threshold, the preset grayscale change threshold and the preset displacement range threshold to retain the vectors that meet the conditions. After obtaining the positive pixel displacement, the displacement calculation module calls the transformation matrix output by the image alignment module to correct the positive pixel displacement, eliminate the influence of the overall movement of the ECU cover, and obtain the effective displacement vector. This effective displacement vector reflects the change of the highlight pixel area within the detection area relative to the workpiece coordinate system, rather than the translational change of the workpiece body, thus providing usable data for subsequent trajectory analysis; Both forward pixel displacement and reverse pixel displacement are represented as two-dimensional displacement vectors in the same detection region coordinate system, and the number of their components is consistent. When the modules perform mutual verification, they take the same pixel position or the same local matching unit as the corresponding object, and determine whether the deviation of the angle between the forward pixel displacement and the reverse pixel displacement in the direction is less than 180 degrees and whether the difference in their amplitude is less than the preset amplitude error threshold. Only vectors that meet the above two error threshold conditions at the same time are retained. The preset grayscale change threshold is used to constrain the grayscale change range of pixels participating in displacement calculation, and the preset displacement range threshold is used to constrain the displacement amplitude range. Both are only applied to pixel-level displacement filtering and are not used in conjunction with the overall displacement threshold. The transformation matrix output by the image alignment module is used to represent the inter-frame transformation relationship between two adjacent frames in a unified coordinate system; When the transformation matrix includes rotation angle Translation components Two-dimensional homogeneous transformation matrix At this time, the displacement calculation module first maps the positive pixel displacement coordinates to the workpiece coordinate system, and the specific formula is as follows: in, It is a two-dimensional homogeneous transformation matrix. It is a positive pixel displacement vector. and These are the horizontal and vertical components of the vector, respectively; The effective displacement vector after mapping , and These are the components of the effective displacement vector in the transverse and longitudinal directions, respectively; Since the displacement vector is translation invariant, it is padded with zeros at the end during homogeneous coordinate mapping, and its operation is only subject to the two-dimensional homogeneous transformation matrix. The rotational component in the model naturally isolates the translational component introduced by the overall movement of the workpiece from the rotational component in the model, eliminating the need for additional displacement subtraction operations. If the transformation matrix fails to be solved, the matrix parameters are missing, or the corrected displacement vector exceeds the preset displacement range threshold, the corresponding vector is discarded and will not be included in the subsequent trajectory analysis.

[0027] In one alternative approach, the detection feature data includes the degree of consistency between the direction of the effective displacement vector and the trajectory tangent direction, the coverage of the effective displacement vector, the amount of local pixel change, the sustained occurrence rate of specular highlights, and the specular overlap rate between adjacent frames; the trajectory analysis module calculates the detection feature data within each segment detection window respectively; Among them, the local pixel change is calculated only using the effective displacement vectors whose corresponding pixel's preset neighborhood window is located within the detection area. When the number of effective displacement vectors is lower than the preset number threshold, the corresponding local pixel change is marked as invalid. When the coverage of the effective displacement vector is lower than the preset coverage threshold, the corresponding directional consistency data is not included in the judgment of sealant continuity.

[0028] In this embodiment, data completeness refers to the state in which the number of effective displacement vectors used to calculate the degree of consistency of direction, the amount of local pixel change and the sustained occurrence rate of specular highlights, the number of adjacent frames involved in the calculation and the specular pixel extraction results in the current window all meet the preset conditions simultaneously; when any of the above conditions are not met, the data completeness of the window is determined to be insufficient and will not participate in the subsequent connectivity calculation. In this embodiment, the trajectory analysis module calculates each segment detection window using the set of effective displacement vectors within the same window; The tangent direction of the trajectory is obtained by fitting and differentiating the discrete trajectory points of the standard sealant trajectory within the current segmented detection window, or by determining the direction of the line connecting adjacent discrete trajectory points. The degree of directional consistency is statistically determined based on whether the angle between the effective displacement vector direction and the trajectory tangent direction falls within the allowable angle range. If the proportion of vectors within the window that meet the allowable angle range reaches the preset directional consistency threshold, the degree of directional consistency of the window is recorded as valid. The coverage rate of the effective displacement vector is calculated as the ratio of the length of the trajectory covered by the effective displacement vector within the window to the total trajectory length of the window. When the ratio is lower than the preset coverage rate threshold, the directional consistency of the window is not involved in the judgment of the sealant continuity. Local pixel changes are only counted for effective displacement vectors that are located within the detection area and whose preset neighborhood windows are completely within the detection area, thus avoiding deviations at the window edges due to missing neighborhoods. When the number of valid displacement vectors that can be included in the statistics is lower than the preset threshold, the local pixel change of the window is directly marked as invalid; the highlight persistence rate is calculated as the ratio of the number of frames that meet the highlight pixel number threshold in the current window to the total number of frames. If the total number of frames is insufficient, the cumulative ratio of the frames already collected for the current workpiece is used and the window is marked as pending supplementation. The calculation will be repeated after reshooting. Data completeness is determined by three factors: the number of effective displacement vectors within the window, the number of adjacent frames included in the statistics, and the results of specular pixel extraction. When the number of effective displacement vectors is insufficient, the number of adjacent frames is missing, or the results of specular pixel extraction are empty, any one of these factors will reduce the data completeness of the window to insufficient. The window is allowed to proceed to the subsequent connectivity calculation only when all three conditions are met. Among the three conditions, the number of effective displacement vectors is the main criterion, the number of adjacent frames is the auxiliary criterion, and an invalid flag is directly triggered when the highlight pixel extraction result is empty, thereby avoiding misjudgment of continuity in the case of insufficient samples. The length of the trajectory covered by the effective displacement vector within the window is calculated by statistically analyzing the discrete trajectory points within the segmented detection window according to the standard sealant trajectory: first, the trajectory within the window is discretized into multiple sequentially arranged trajectory points according to the preset trajectory sampling interval; then, the presence or absence of at least one effective displacement vector within the preset neighborhood of each trajectory point is used as the criterion for whether the trajectory point is covered; the number of trajectory points determined to be covered is multiplied by the preset trajectory sampling interval to obtain the length of the covered trajectory. The total trajectory length of the window is the sum of the distances between all adjacent trajectory points within the segmented detection window. The coverage rate is obtained by dividing the length of the covered trajectory by the total trajectory length of the window. When the total trajectory length of the window is 0, the current window is directly marked as invalid and does not participate in the coverage rate calculation. The local pixel change is represented by the window average of the absolute value of the gray-level difference. Specifically, taking the pixel position corresponding to each effective displacement vector as the center, a preset neighborhood window of the same size is extracted in the two consecutive frames, and the local change value corresponding to the effective displacement vector is calculated. Through the formula: in, For the first Local change values ​​corresponding to each effective displacement vector The index of the effective displacement vector; Define the pixel width and height of the preset neighborhood window; These are the horizontal and vertical pixel coordinates within the preset neighborhood window, respectively. and These represent the frame numbers of the current frame and the previous frame, respectively. and These are the grayscale values ​​at corresponding coordinate positions in the preceding and following frames; further, the local pixel change within the current segment detection window is calculated. The formula is as follows: in, This represents the total number of valid displacement vectors within the current segmented detection window; the size of the preset neighborhood window is set according to the principle of not exceeding the boundary of the detection area and covering continuous pixels around the pixel position, and remains consistent for the same batch of workpieces; The preset orientation consistency threshold, preset coverage threshold, and preset quantity threshold are all obtained from the statistics of normal continuous sealing trajectories of standard samples. The preset orientation consistency threshold is taken as the lower boundary of the proportion of orientation consistency vectors in normal samples, the preset coverage threshold is taken as the lower boundary of trajectory coverage in normal samples, and the preset quantity threshold is taken as the minimum effective displacement vector number that can stably calculate the local pixel change. This allows the orientation consistency, coverage, and local pixel change to be repeatedly calculated within the same segmented detection window.

[0029] In one alternative approach, the trajectory analysis module predicts the position of the current frame's highlight pixel region in the next frame based on the effective displacement vector, and calculates the ratio of the overlapping area between the predicted position and the highlight pixel region in the next frame to obtain the highlight overlap rate between adjacent frames; the highlight persistence rate is determined by statistically analyzing the proportion of frames in each segment detection window where the number of highlight pixels is greater than or equal to a preset highlight pixel number threshold to the total number of frames. The trajectory analysis module determines the reflection type label based on whether the sustained occurrence rate of highlights and the overlap rate of highlights in adjacent frames meet preset thresholds.

[0030] In this embodiment, the trajectory analysis module adopts a linear translation model, and its calculation formula is as follows: in, These are the pixel coordinates within the detection area of ​​the current frame; This corresponds to the effective displacement. To predict the position of the current frame's highlight pixel region in the next frame, the predicted position is compared with the highlight pixel region of the next frame, and the overlap area ratio is calculated to obtain the highlight overlap rate between adjacent frames. This overlap rate is used to reflect whether the highlight pixel regions of adjacent frames have a traceable relationship. The sustained occurrence rate of highlights is obtained by counting the number of frames in each segment detection window where the number of highlight pixels meets the preset threshold for the number of highlight pixels, and then calculating the proportion of the total number of frames. This ratio is used to reflect whether the highlight continues to appear in multiple sampling frames; the trajectory analysis module determines the reflection type label based on whether the highlight persistence rate and the highlight overlap rate of adjacent frames meet the preset thresholds, so as to distinguish between stable highlights and short-term interference highlights, and provide a basis for subsequent continuity judgment; The reflection type is divided into three categories: stable reflection, fluctuating reflection, and interfering reflection. Among them, when the sustained occurrence rate of the highlight and the overlap rate of the highlight in adjacent frames both meet the preset threshold, it is marked as stable reflection. If only one of the preset thresholds is met, it is marked as fluctuating reflection; if neither of the preset thresholds is met, it is marked as interfering reflection. The preset threshold is determined by the statistical results of the standard sample under three conditions: normal sealing, partial light blurring, and obvious discontinuity, and the boundary values ​​that can be stably distinguished for the three types of samples are taken respectively.

[0031] In one alternative approach, the continuity judgment module uses each segmented detection window as a judgment node. When the coverage of the effective displacement vector and the sustained occurrence rate of the highlight meet the preset confidence thresholds, the corresponding judgment node is marked as a confidence node. For adjacent reliable nodes that meet the preset threshold conditions of direction consistency, local pixel change, specular persistence rate and adjacent frame specular overlap rate, a connection relationship is established based on the angle and magnitude difference of the direction change of the average displacement vector between the two nodes. Windows that have not established connections or do not meet the confidence threshold are marked as continuous abnormal windows, and the connectivity of the sealant is calculated based on the proportion of node connections and the penalty coefficient of the length of continuous abnormal windows. For the standard sealant trajectory of the closed loop, the continuity judgment module also judges the connection relationship between the start and end segment detection windows.

[0032] In this embodiment, the proportion of node connections is calculated as the ratio of the number of adjacent window pairs with established connections to the total number of adjacent window pairs participating in the judgment. When the number of all adjacent window pairs involved in the judgment is 0, the current trajectory is directly marked as insufficient data, and the connectivity rate is not calculated; the penalty coefficient is determined according to the length of the consecutive abnormal windows and is limited to the range of 0 to 1. In this embodiment, the continuity judgment module processes the judgment nodes in the following order: first, check the data completeness, then check the coverage and the sustained occurrence rate of highlights, and then check the consistency of direction and the overlap rate of highlights in adjacent frames. If any of the preconditions are not met, the subsequent connection relationship will not be calculated, and the node will be directly transferred to the continuous anomaly window. For adjacent trusted nodes, the module compares the angle between the directions of the average displacement vectors of the two nodes. If the angle exceeds the allowable range, the connection relationship between the two nodes will not be established. If the included angle meets the requirements, the amplitude difference is compared. A connection is established only when the amplitude differences fall within the allowable range. This establishment process is executed pair by pair in window order. The connection result of the previous pair of nodes does not affect the threshold determination of the next pair of nodes, but it will affect the proportion of the number of connected nodes in the final connectivity. For continuous abnormal windows, the module applies a penalty coefficient in segments according to their continuous length. The longer the continuous length, the larger the penalty coefficient. When the length of the continuous abnormal window reaches the preset upper limit, the trajectory segment is directly regarded as a discontinuous risk segment, and the local improvement results of adjacent windows are no longer accumulated. For the closed-loop standard sealant trajectory, when the module calculates the connection relationship between the start and end point segment detection windows, it prioritizes the consistency of the average displacement vector direction between the start and end point windows for closed-loop verification. If the closed-loop verification fails, even if the connectivity of the intermediate window meets the requirements, the entire trajectory will still be marked as a suspected sealant break, thus ensuring that the consistency of the start and end of the closed-loop trajectory is not covered by local continuous segments. The average displacement vector is calculated based on the average of the components of all valid displacement vectors within the current node. First, the horizontal and vertical components of each valid displacement vector are averaged separately, and then the average displacement vector of the node is formed by the two averaged components. When the number of valid displacement vectors within a node is lower than a preset threshold, the node does not calculate the average displacement vector and is directly determined as an untrusted node. The angle between the directions of two nodes is determined by the angle between their average displacement vectors, the magnitude difference is determined by the absolute value of the difference in the magnitudes of their average displacement vectors, and the allowable range and allowable interval are determined by the statistical boundaries of adjacent windows under the continuous sealed state of the standard sample. The length of consecutive abnormal windows is counted according to the number of abnormal windows that appear consecutively along the standard sealant trajectory. When the length of each segment detection window is the same, the length of consecutive abnormal windows is represented by the number of consecutive abnormal windows. When the length of each segment detection window is different, the length of consecutive abnormal windows is represented by the sum of the lengths of the corresponding window trajectories. The penalty coefficient is determined by a tiered assignment method: when the length of the consecutive abnormal window is within the first length range, the first penalty value is taken; when it is within the second length range, the second penalty value is taken. The larger the length of the consecutive abnormal window, the larger the corresponding penalty value, and the penalty value of the later tier is not less than the penalty value of the previous tier. The connectivity rate is obtained by combining the proportion of the number of connected nodes with the penalty coefficient. Specifically, the ratio of the number of adjacent window pairs with established connections to the total number of adjacent window pairs involved in the judgment is calculated to obtain the basic connectivity ratio. Then, the basic connectivity ratio is subtracted according to the penalty coefficient corresponding to the length of consecutive abnormal windows to obtain the connectivity rate. When there are no consecutive abnormal windows, the penalty coefficient is 0. When there are multiple consecutive abnormal windows on the same trajectory, the penalty coefficient corresponding to the longest consecutive abnormal window is used first for connectivity calculation. When there are two or more consecutive abnormal windows of the same length, the penalty coefficient corresponding to the consecutive abnormal window with the smallest distance from the start and end connection of the closed loop is used for calculation to ensure that abnormalities at the end of the closed loop trajectory are given priority in the judgment.

[0033] In one alternative, the re-inspection control module determines the test results as sealant continuity, suspected sealant breakage, confirmed sealant breakage, or invalid data based on data validity marking, the proportion of trusted nodes, the length of the continuous abnormal window, and the consistency of the results before and after re-shooting. Among these, the invalid data state triggers ECU cover repositioning, industrial camera exposure adjustment, or output of manual re-inspection prompt information; the suspected sealant breakage state triggers in-situ re-shooting. If the length of consecutive abnormal windows within the same trajectory segment exceeds a preset length threshold, and consecutive abnormal windows still appear in the same trajectory segment after reshooting, output confirmation of glue breakage status.

[0034] In this embodiment, the consistency of the results before and after the reshoot is determined by whether the position deviation and length change of the abnormal window within the same trajectory segment simultaneously meet the preset conditions; if the position deviation of the abnormal window exceeds the preset position deviation threshold or the length of the abnormal window is lower than the preset length threshold after the reshoot, it will not be upgraded to confirm the glue breakage. In this embodiment, the re-inspection control module performs the judgment in a fixed order: first, it reads the data validity mark, then it counts the proportion of trustworthy nodes, reads the length of the continuous abnormal window, and compares the consistency of the results before and after the re-shoot. If the data validity is marked as invalid, the current workpiece will directly output the invalid data status and will no longer enter the judgment of the proportion of trusted nodes and the length of the continuous anomaly window; If the data is marked as valid and the proportion of trusted nodes reaches the preset trusted proportion threshold, then the continuous anomaly window length judgment continues; if the proportion of trusted nodes does not reach the preset trusted proportion threshold, then the current result is recorded as suspected glue breakage and in-situ reshoot is triggered. During the continuous abnormal window length judgment stage, if there are no continuous abnormal windows in the same trajectory segment, and if the length of the continuous abnormal window is less than or equal to the preset length threshold, the current result will be maintained as the first preset safe state between sealant continuity and suspected sealant breakage. If the length of the continuous abnormal window is greater than the preset length threshold, then the consistency judgment of the results before and after the reshoot will be entered; the first preset safety state means that when the current data is valid and the proportion of trusted nodes meets the requirements, the sealant continuity will be maintained first, and when the current data is valid but the proportion of trusted nodes is insufficient, the sealant breakage will be maintained. The consistency of the results before and after the reshoot is determined by the positional deviation and length change of the abnormal window within the same trajectory segment. The positional deviation is calculated by the distance along the standard sealant trajectory between the center position of the abnormal window before and after the reshoot, and the length change is calculated by the absolute value of the difference between the lengths of the abnormal window before and after the reshoot. The results before and after the reshoot are considered consistent only if the positional deviation does not exceed the preset positional deviation threshold and the length of the abnormal window after the reshoot is not less than the preset length threshold. If the abnormal window does not reappear on the same trajectory segment after the reshoot, or if the abnormal window has moved to another trajectory segment, the reshoot result is considered inconsistent and is not upgraded to confirm the glue breakage. The subsequent actions corresponding to invalid data are executed according to the invalid reason: when the invalid reason is alignment failure, the ECU cover is repositioned; when the invalid reason is insufficient highlight pixels or exposure saturation, the industrial camera exposure is adjusted; when the same workpiece is still invalid after reaching the preset number of retakes, a manual re-inspection prompt message is output; the suspected glue breakage state corresponds to in-situ retake, and after the retake, data verification, trajectory analysis and continuity judgment are re-executed. After confirming the glue breakage status output, the re-inspection control module sends a rejection permission signal to the rejection mechanism and latches the detection result corresponding to the workpiece number until the workpiece leaves the inspection station before releasing it; the preset number of retakes, preset confidence ratio threshold, preset length threshold and preset position deviation threshold are all determined by the statistical boundaries of the standard sample and the known glue breakage sample.

[0035] In one alternative, the data processing terminal also runs a parameter setting module. The parameter setting module generates detection parameters based on the results of manual review or standard sample testing, and when the detection misjudgment rate exceeds the preset misjudgment rate threshold within a preset statistical period, the detection parameters are rolled back to the previous parameter version.

[0036] In this embodiment, in addition to running modules related to image acquisition, alignment, highlight extraction, displacement calculation, data verification, trajectory analysis, continuity judgment, and re-inspection control, the data processing terminal also runs a parameter setting module. The input source for the parameter setting module is the result of manual verification or the test result of the standard sample. The module generates detection parameters based on these parameters and saves the parameters in a versioned manner. If the false detection rate exceeds the preset false detection rate threshold within the preset statistical period, the parameter setting module will revert the current detection parameters to the previous parameter version to ensure that subsequent detections still use the verified parameter configuration. The parameter rollback action is executed by the data processing terminal and simultaneously applied to the image acquisition, alignment, highlight extraction, displacement calculation, and continuity judgment processes, enabling the system to continue operating with the rolled-back parameters in subsequent cycles.

Claims

1. An ECU housing cover sealing detection system based on image recognition, characterized in that, It includes an industrial camera, a light source assembly, a rejection mechanism, and a data processing terminal that is communicatively connected to the industrial camera and the rejection mechanism. The data processing terminal operates as follows: The image acquisition module controls the industrial camera to continuously acquire a preset number of image sequences of the same ECU housing cover when the light source component is turned on; The image alignment module extracts the positioning reference based on the template data, aligns each frame of image to the same coordinate system, outputs the aligned image and alignment error, and matches the standard sealant trajectory. The highlight extraction module extends a preset pixel distance along both sides of the trajectory to form a strip-shaped region, performs morphological expansion on the extracted highlight pixel region within the strip-shaped region to obtain the detection region, and outputs the effective detection region percentage. The displacement calculation module calculates the displacement vector of adjacent frame pixels and the overall displacement of the ECU cover within the detection area. After coordinate transformation, the overall displacement of the ECU cover is eliminated to obtain the effective displacement vector. The data verification module verifies the alignment error, overall displacement, reliability of the effective displacement vector, and proportion of the effective detection area, and outputs data validity labels. The trajectory analysis module divides the trajectory into segmented detection windows and calculates the detection feature data within each window based on the effective displacement vector with data validity markers. The continuity judgment module determines the continuity of the sealant and outputs the detection results based on the detection feature data of adjacent windows and the changes in displacement direction and amplitude between windows; The re-inspection control module controls the rejection mechanism to perform rejection when the glue breaks, and controls the industrial camera to retake the picture when there is an abnormality or invalid data.

2. The ECU housing cover sealing detection system based on image recognition according to claim 1, characterized in that, The system also includes a position sensor that is communicatively connected to the data processing terminal. The image acquisition module continuously acquires image sequences and records the acquisition time interval within a preset time after receiving the ECU cover positioning signal from the position sensor. The displacement calculation module calculates the overall displacement of the ECU housing between two adjacent frames and determines whether the overall displacement is within a preset effective range. When the overall displacement is less than the first displacement threshold or greater than the second displacement threshold, the current image sequence is determined to be invalid and a delayed reshoot is triggered. When the overall displacement is within the preset effective range, the image sequence is sent to the highlight extraction module.

3. The ECU housing cover sealing detection system based on image recognition according to claim 1, characterized in that, The image alignment module identifies at least one of the positioning holes, corner points, or outlines that serve as the positioning reference, calculates the transformation matrix required for image alignment of each frame, and obtains the alignment error by evaluating the image alignment quality based on the alignment error of the positioning reference. When the alignment error of any frame exceeds the preset alignment error threshold, repositioning or reshooting is triggered, the image sequence is marked as invalid and prohibited from entering the continuity judgment process. When the alignment error of any frame is less than or equal to the preset alignment error threshold, the aligned image is output.

4. The ECU housing cover sealing detection system based on image recognition according to claim 1, characterized in that, The highlight extraction module performs binarization processing on the aligned image to extract the highlight pixel region, and filters out abnormal reflective areas within the strip region according to a preset area threshold. The highlight extraction module expands the filtered highlight pixel region outward by a preset number of pixels and intersects it with the strip region to obtain the detection region. When the number of highlight pixels in the strip area is less than the preset highlight pixel number threshold, an invalid detection status is output, and the industrial camera exposure adjustment or reshoot is triggered.

5. The ECU housing cover sealing detection system based on image recognition according to claim 3, characterized in that, The displacement calculation module calculates the forward pixel displacement and the reverse pixel displacement in the detection area of ​​two adjacent frames of images, and retains the vectors that meet the conditions according to the preset error threshold, the preset grayscale change threshold and the preset displacement range threshold. The displacement calculation module corrects the positive pixel displacement based on the transformation matrix of the image alignment module to obtain an effective displacement vector that eliminates the overall displacement of the ECU housing.

6. The ECU housing cover sealing detection system based on image recognition according to claim 1, characterized in that, The detection feature data includes the degree of consistency between the direction of the effective displacement vector and the trajectory tangent direction, the coverage of the effective displacement vector, the amount of local pixel change, the sustained occurrence rate of highlights, and the overlap rate of highlights in adjacent frames. The trajectory analysis module calculates the detection feature data within each segment detection window; wherein, the local pixel change is calculated only using the effective displacement vectors whose corresponding pixel preset neighborhood windows are all located within the detection area; when the number of effective displacement vectors is lower than a preset number threshold, the corresponding local pixel change is marked as invalid. When the coverage of the effective displacement vector is lower than the preset coverage threshold, the corresponding directional consistency data is not included in the judgment of sealant continuity.

7. The ECU housing cover sealing detection system based on image recognition according to claim 6, characterized in that, The trajectory analysis module predicts the position of the current frame's highlight pixel region in the next frame based on the effective displacement vector, and calculates the ratio of the overlapping area between the predicted position and the highlight pixel region in the next frame to obtain the highlight overlap rate of the adjacent frames. The sustained occurrence rate of highlights is determined by statistically analyzing the proportion of frames in each segment detection window where the number of highlight pixels is greater than or equal to a preset threshold for the number of highlight pixels to the total number of frames. The trajectory analysis module determines the reflection type label based on whether the sustained occurrence rate of the highlight and the overlap rate of the highlight in adjacent frames meet preset thresholds.

8. The ECU housing cover sealing detection system based on image recognition according to claim 6, characterized in that, The continuity judgment module uses each segmented detection window as a judgment node. When the coverage of the effective displacement vector and the sustained occurrence rate of the highlight meet the preset confidence thresholds, the corresponding judgment node is marked as a confidence node. For adjacent reliable nodes that meet the preset threshold conditions of direction consistency, local pixel change, highlight persistence rate and adjacent frame highlight overlap rate, a connection relationship is established based on the angle and magnitude difference of the direction change of the average displacement vector between the two nodes. Windows that have not established connections or do not meet the trust threshold are marked as continuous abnormal windows, and the connectivity of the sealant is calculated based on the proportion of node connections and the penalty coefficient of the length of continuous abnormal windows. For a closed-loop standard sealant trajectory, the continuity judgment module also determines the connection relationship between the start and end point segment detection windows.

9. The ECU housing cover sealing detection system based on image recognition according to claim 8, characterized in that, The re-inspection control module determines the detection results as sealant continuity, suspected sealant breakage, confirmed sealant breakage, or invalid data based on the data validity marker, the proportion of reliable nodes, the length of the continuous abnormal window, and the consistency of the results before and after the re-shoot. Among them, invalid data status triggers ECU cover repositioning, industrial camera exposure adjustment, or output of manual re-inspection prompts; suspected glue breakage status triggers in-situ reshooting. If the length of consecutive abnormal windows within the same trajectory segment exceeds a preset length threshold, and consecutive abnormal windows still appear in the same trajectory segment after reshooting, output confirmation of glue breakage status.

10. The ECU housing cover sealing detection system based on image recognition according to claim 1, characterized in that, The data processing terminal also runs a parameter setting module. The parameter setting module generates detection parameters based on the results of manual review or standard sample testing, and when the detection misjudgment rate exceeds the preset misjudgment rate threshold within a preset statistical period, the detection parameters are rolled back to the previous parameter version.

Images