A CCD vision detection-based automobile wire harness quality identification system and method thereof

Abstract

The application provides a kind of automobile wire harness quality identification system and method based on CCD vision detection, it is related to automatic detection technical field.The system includes: image acquisition module, for collecting 2D image;Image processing module is used to identify two-dimensional defect feature;Three-dimensional topography measurement module is used to obtain 3D geometric parameter;And ultrasonic wave compactness detection module is used to non-destructively detect internal compression joint compactness.The system also includes a multi-modal data fusion and identification module, which is configured to: fuse the two-dimensional defect feature, 3D geometric parameter and compression joint compactness data to generate a multi-dimensional state vector;And based on the time series analysis of the multi-dimensional state vector, generate the identification result of wire harness quality and process state early warning signal.The application also provides a method for applying the system.The application realizes multi-dimensional, non-destructive online full detection.

Description

Technical Field

[0001] This invention relates to the field of automated inspection technology, and in particular to a CCD vision inspection-based automotive wiring harness quality identification system and method. Background Technology

[0002] In the modern automotive industry, automotive wiring harnesses serve as the nervous system of the entire vehicle, undertaking critical functions of power transmission and data signal control. The reliability and safety of the wiring harness directly affect the performance of the entire vehicle and the safety of passengers. The crimping process of the wiring harness terminals is the core step in mechanically and electrically connecting the wire cores to the metal terminals. The quality of the crimping, including its mechanical strength and electrical continuity, is a key factor determining the long-term reliability of the wiring harness assembly and even the entire vehicle.

[0003] Currently, the main testing methods used in the industry for the crimping quality of wire harness terminals include: Offline destructive testing: This is the most traditional and widely accepted method of quality verification. Specific methods include pull-out force testing, which measures the force required to pull the wire out of the terminal; and crimp cross-section analysis, which involves cutting, polishing, and microanalyzing the crimped terminal to examine its internal crimp morphology and compactness.

[0004] Manual visual inspection: During the production process, experienced quality inspectors visually inspect the finished products. This is usually done by sampling and mainly to check for obvious surface defects such as damaged wire insulation or deformed terminals.

[0005] Online 2D Vision Inspection Systems: With the development of industrial automation, many production lines are equipped with 2D vision systems based on industrial cameras. These systems can perform full online inspections and are mainly used to detect visible surface defects, such as the presence or absence of materials (e.g., waterproof sealing rings), whether the wire color is correct, and whether the wire core (copper wire) is properly exposed outside the insulation layer.

[0006] Crimping height measurement: Crimping height is widely recognized as one of the most critical geometric indicators for judging crimping quality. Currently, this parameter is mainly measured manually by sampling offline using specialized tools such as micrometers.

[0007] While the above methods ensure product quality to some extent, their inherent limitations become increasingly apparent when facing increasingly stringent zero-defect quality targets in the automotive industry. First, while offline destructive testing (such as pull-out force testing and cross-sectional analysis) is accurate, its destructive nature means that it can only be used for small-batch sampling inspections (such as when changing molds every shift or batch), and cannot achieve 100% online full inspection. Therefore, there is a risk that occasional defective products will flow into downstream products.

[0008] Secondly, manual visual inspection is inefficient and highly dependent on the experience, physical condition, and sense of responsibility of the inspectors. It is easily affected by subjective factors, resulting in inconsistent testing standards and a high rate of missed detections.

[0009] Secondly, and most importantly, traditional online 2D vision systems have dimensional limitations. 2D vision can only detect surface defects. For the two most critical indicators of crimping quality, (A) the crimping density within the crimping area (e.g., whether there are gaps between wire strands, whether the crimping is loose, or whether there are missing wire strands); and (B) key 3D geometric parameters (such as the crimping height CCH that determines whether the crimping is over-crimped or under-crimped), 2D vision systems are completely powerless.

[0010] Over-crimping can cause partial wire breakage due to excessive pressure, leading to a decrease in electrical performance; while under-crimping will result in loose crimping and easy cable detachment. Both are serious functional defects. Although crimping height is a key indicator of these defects, traditional micrometer measurement methods are difficult to achieve online, high-speed, and accurate full inspection.

[0011] Finally, all of the above methods are passive defect removal mechanisms. They can only identify defects after they have occurred, and cannot monitor changes in the production process itself, such as tool wear of the pressing die or other process drifts. When the die begins to wear slowly, the resulting minute drifts in quality parameters will not be discovered until they accumulate to a certain extent and produce a batch of defective products, which has already caused huge waste of materials and production capacity.

[0012] In summary, there is an urgent lack of an integrated, non-destructive, online full inspection system in the existing technology that can simultaneously detect the surface, geometry, and internal (density) properties, and it is even more impossible to achieve predictive maintenance of the condition of key process equipment such as pressing molds. Summary of the Invention

[0013] The purpose of this invention is to provide an automotive wiring harness quality identification system and method based on CCD vision inspection, in order to solve the problems pointed out in the background art, that existing inspection methods can only detect surface defects, cannot non-destructively assess internal density and 3D geometry, and cannot predict process status.

[0014] In a first aspect, embodiments of the present invention provide an automotive wiring harness quality identification system based on CCD vision inspection, the system comprising: An image acquisition module configured to acquire 2D images of the automotive wiring harness terminals; An image processing module is connected to the image acquisition module, and the image processing module is configured to identify two-dimensional defect features of the terminal based on the 2D image; A three-dimensional topography measurement module is configured to acquire the 3D geometric parameters of the terminal crimping area; An ultrasonic tightness testing module configured to non-destructively test the crimp tightness inside the terminal; and A multimodal data fusion and recognition module is connected to the image processing module, the three-dimensional topography measurement module, and the ultrasonic compactness detection module, respectively. The multimodal data fusion and recognition module is configured as follows: The two-dimensional defect features, the 3D geometric parameters, and the press-fit compaction are integrated to generate a multi-dimensional state vector; and Based on the time series analysis of the multidimensional state vector, the identification results of the wire harness quality and the early warning signal of the process status are generated.

[0015] Optionally, the image processing module includes: The material has at least one of the following: a testing tool for the presence or absence of a wire sheath crimping test tool, and a testing tool for the exposure of wire cores on the left and right sides.

[0016] Optionally, the image processing module further includes: A color recognition tool is configured to detect exposed copper wires at the terminal crimping point by color recognition.

[0017] Optionally, the image acquisition module includes: A front-end CCD camera is disposed on one side of the terminal; and A rear-end CCD camera is located on the other side of the terminal.

[0018] Optionally, the system further includes: A parameter management module is connected to the image processing module, and the parameter management module is configured to enable rapid switching of product models through a copy loading mechanism.

[0019] Optionally, the system further includes: A result display and alarm module, which is connected to the multimodal data fusion and recognition module, is used to receive and display the recognition results and the process status early warning signal.

[0020] Optionally, the three-dimensional topography measurement module includes: A polarization structured light scanner or a laser triangulation instrument; The three-dimensional topography measurement module is configured to: acquire three-dimensional point cloud data of the terminal crimping area, and calculate the 3D geometric parameters based on the three-dimensional point cloud data, wherein the 3D geometric parameters include crimping height and / or flare size.

[0021] Optionally, the ultrasonic tightness detection module includes: A pair of ultrasonic transducers integrated into the terminal crimping station; The multimodal data fusion and recognition module is further configured to quantify the compression tightness based on the ultrasonic compression wave amplitude attenuation value measured by the ultrasonic transducer.

[0022] Optionally, the multimodal data fusion and recognition module includes: A long short-term memory autoencoder; and A cumulative sum control chart analysis unit; The long short-term memory autoencoder is configured to generate a reconstruction error based on the multidimensional state vector. The cumulative sum and control chart analysis unit is configured to monitor the time series of the reconstruction error to identify process drifts indicating wear of the pressing die and generate the process status warning signal.

[0023] Secondly, embodiments of the present invention provide a method for identifying automotive wiring harness quality based on CCD visual inspection. The method is applied to the system described in any one of the first aspects, and includes the following steps: (a) Acquire a 2D image of the terminal using the image acquisition module, and process the 2D image using the image processing module to obtain the two-dimensional defect features; (b) Obtain the 3D geometric parameters of the terminal crimping area using the three-dimensional topography measurement module; (c) Obtaining the crimping tightness data inside the terminal using the ultrasonic tightness testing module; and (d) Through the multimodal data fusion and recognition module: (d1) Integrate the two-dimensional defect features, the 3D geometric parameters, and the press-fit compaction data to generate a multi-dimensional state vector; and (d2) Based on the time series analysis of the multidimensional state vector, generate the identification result of the wire harness quality and the early warning signal of the process status.

[0024] The present invention has achieved the following beneficial effects: This invention overcomes the major drawback of the prior art, which is that destructive testing cannot achieve 100% online full inspection. By integrating CCD vision (surface inspection), three-dimensional topography measurement (geometry measurement), and ultrasonic testing (internal flaw detection) at a single workstation, this invention provides a solution for online, non-destructive, full-coverage inspection of all key dimensions (surface, geometry, and internal) of wire harness crimping quality.

[0025] This invention overcomes the dimensional limitations of 2D vision systems in the prior art, which cannot detect internal and 3D geometric defects. The 3D topography measurement module of this invention (as described in Example 7) can use laser triangulation to calculate key 3D geometric parameters such as crimp height (CCH) and flare size online and accurately, thereby accurately identifying over-crimped or under-crimped defects that cannot be detected by 2D vision. Simultaneously, the ultrasonic tightness detection module of this invention (as described in Example 8) can penetrate metal terminals and non-destructively quantify their internal crimp tightness, thereby identifying serious defects such as internal looseness, voids, or missing wire cores that cannot be detected even by 3D measurement.

[0026] This invention addresses the critical weakness of existing technologies where all detection methods are unable to predict process drift (such as mold wear). The invention is not merely a simple aggregation of detection modules; its core lies in a multimodal data fusion and recognition module (as described in Example 9). This module fuses heterogeneous data from 2D, 3D, and ultrasonic sources into a high-dimensional, multi-dimensional state vector. It innovatively employs a Long Short-Time Memory Autoencoder (LSTM-AE) and a Cumulative Sum Control Chart (CUSUM) analysis unit to perform time-series analysis on the reconstruction error of this vector. This analysis mechanism can extremely sensitively capture minute process drifts caused by factors such as mold wear, proactively issuing process status warnings before these drifts lead to batch defects. This achieves an industrial upgrade from traditional passive quality control (QC) to proactive predictive maintenance (PdM), significantly improving production yield and reducing substantial economic losses caused by batch scrap and equipment downtime.

[0027] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.

[0028] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0029] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a schematic diagram of an automotive wiring harness quality identification system based on CCD vision inspection in an embodiment of the present invention. Figure 2 This is a flowchart of a method for identifying automotive wiring harness quality based on CCD vision inspection, as described in an embodiment of the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in further detail below with reference to the accompanying drawings and specific embodiments. The preferred embodiments described herein are merely illustrative and not intended to limit the scope of the invention.

[0031] Those skilled in the art will understand that, in this invention, the terms "connected," "configured," etc., should be interpreted broadly. For example, "connected" can be understood as a fixed connection, a detachable connection, or an integral connection; "configured" can be understood as being specifically designed or programmed to perform a particular function. In the description of this invention, references to one embodiment or several embodiments do not necessarily refer to the same embodiment.

[0032] Example 1: This example combines Figure 1 This paper describes the overall architecture and working principle of an automotive wiring harness quality identification system based on CCD vision inspection. Figure 1 As shown, this system is designed for integration into automated automotive wiring harness production lines, typically located adjacent to the terminal crimping station. The system mainly includes: an image acquisition module 101, an image processing module 102, a three-dimensional topography measurement module 103, an ultrasonic compactness detection module 104, and a multimodal data fusion and recognition module 105.

[0033] In a typical workflow, once a vehicle wiring harness terminal to be tested is delivered to the testing station and precisely positioned, the system's top-level controller will trigger a series of near-synchronous testing actions.

[0034] First, the image acquisition module 101 is triggered to acquire high-resolution 2D images of key parts of the terminal. These image data are immediately sent to the image processing module 102. Based on preset parameters (as described in Examples 2 and 3), the image processing module 102 analyzes the 2D images at high speed to extract key two-dimensional defect features, such as the presence or absence of material, the condition of wire insulation crimping, or exposed copper wire.

[0035] Almost simultaneously, the 3D topography measurement module 103 is triggered. This module (as described in Embodiment 7) employs non-contact optical measurement technology (e.g., laser triangulation) to perform high-speed scanning of the terminal crimping area to obtain high-density 3D point cloud data of the surface of this area. Subsequently, the module's internal processor processes the point cloud data to calculate key 3D geometric parameters, such as the crimping height and the flare size.

[0036] Similarly, the ultrasonic tightness detection module 104 is also triggered almost simultaneously. The ultrasonic transducer of this module (as described in Embodiment 8) (e.g., integrated on the workstation fixture) is activated, emitting a beam of ultrasonic compression wave that penetrates the terminal crimping area. The receiving transducer then measures the amplitude attenuation value of this compression wave, which is used to quantify the crimp tightness inside the terminal.

[0037] Finally, the outputs of the above three modules—two-dimensional defect features (usually discrete category data) from image processing module 102, 3D geometric parameters (usually continuous measurement values) from three-dimensional topography measurement module 103, and compression compactness (usually continuous attenuation values) from ultrasonic compactness detection module 104—are sent in real time and together to the core of the system, namely the multimodal data fusion and recognition module 105.

[0038] The working principle of the multimodal data fusion and recognition module 105 comprises two levels. The first level is fusion, which is responsible for processing heterogeneous data from different sources and of different types. Through standardized data processing procedures (such as one-hot encoding and normalization), the module fuses these instantaneous measurement results into a unified, high-dimensional mathematical representation, namely a multidimensional state vector. This vector mathematically and completely describes the comprehensive quality state of the current terminal in the three dimensions of surface, geometry, and interior.

[0039] The second aspect is analysis, which is where the advancement of this invention lies. Module 105 maintains a time series consisting of multidimensional state vectors from the past N terminals (e.g., N=1000). Based on the analysis of this time series (as described in Embodiment 9), module 105 is configured to generate two distinct output signals. The first output is the harness quality identification result, i.e., based on the analysis of the current vector (e.g., judging its deviation from normal through a deep learning model, such as reconstruction error), an immediate quality judgment (good or defective) is generated for this terminal. The second output is a process status warning signal, i.e., based on the analysis of the time series (e.g., monitoring whether the reconstruction error is slowly and continuously increasing through a control chart), determining whether the entire production process (e.g., the wear state of the crimping die) is stable. If a small, persistent process drift is detected, the module generates a warning signal, thereby alerting maintenance personnel before a batch of defective products is generated.

[0040] Example 2: This example details the internal structure of the image processing module 102 in Example 1. This module includes various algorithmic tools for identifying two-dimensional defect features.

[0041] The material presence / absence detection tool is configured to confirm the presence of materials (e.g., waterproof seals) that should be present on the terminal assembly. Its working principle is primarily based on binarization and speckle analysis techniques in machine vision. During offline parameter setting, the operator defines a region of interest (ROI) around the material to be tested on a standard image. During online detection, the system extracts the image within this ROI and binarizes it, converting it into a black-and-white image. Subsequently, the system executes a speckle analysis algorithm to find and quantify the number or total area of ​​specks of a specific color (e.g., black) within the ROI. If the calculated number or area of ​​specks does not meet the preset standard (e.g., the number is zero), the system immediately determines that the material is missing.

[0042] The wire insulation (sheath) inspection tool is configured to detect whether the insulation layer (sheath) of a wire has been incorrectly crimped within a conductive wirecrimp, a serious defect that can lead to electrical connection failure. It works by locating the conductive area and searching for features of the wire insulation within that area. The operator precisely selects the conductive wire core crimp area of ​​the terminal on a standard image, defining it as a Forbidden Region of Interest (ROI). During online inspection, the system applies an edge detection algorithm (e.g., the Canny operator) to delineate the outline of objects in the image. The system then checks whether an edge belonging to the wire insulation appears within the Forbidden Region, or (optionally, in conjunction with the tool of Example 3) whether a large area of ​​wire insulation color appears. If such features are detected, the system determines that a wire insulation crimp defect exists.

[0043] The left and right side wire core exposure detection tool is configured to detect whether the wire core (copper wire) at the end of the conductor is properly exposed from the front end (flared area) and rear end (brushed area) of the terminal crimping area, as required by the process. The tool defines two independent ROIs at the front and rear ends of the terminal and measures the length or area of ​​the exposed copper wire (typically appearing as a bright feature under high contrast) through binarization and spot analysis. For the flared ROI, the system checks for the presence of bright spots; if none are found, the wire core is deemed not to be properly inserted. For the brushed area ROI, the system measures the total length of the bright spots and compares it to preset process specifications (e.g., minimum 0.5 mm, maximum 1.5 mm). If the length is too short or too long, the system will classify it as a defect.

[0044] Example 3: This example details an optional advanced tool in the image processing module 102, namely a color recognition tool. This tool is specifically designed to detect certain types of defects, such as whether there are exposed strands of wire (copper wire) in the insulation crimping area, which could lead to a short circuit.

[0045] In industrial environments, fluctuations in workshop lighting conditions can cause traditional RGB color model recognition to fail. To overcome this technical challenge, the color recognition tool in this embodiment is configured to perform analysis in the HSV (hue, saturation, brightness) color space, which is insensitive to changes in light intensity.

[0046] The tool works as follows: First, after the system acquires an RGB image, the color recognition tool converts it into an HSV color model image at the software level. During the offline parameter setting phase, the operator uses the color picker tool to click on the copper wire area of ​​the standard image. The system automatically analyzes the HSV value of that area, and the operator defines a (H, S, V) threshold range (i.e., color model) representing the color of the copper wire based on this. Simultaneously, the operator defines areas on the image that require focused monitoring (e.g., the ROI of the insulation crimping area).

[0047] During online inspection, the system acquires images and converts them to HSV color space. The system analyzes only the pixels within the Region of Interest (ROI), iterating through and counting how many pixels within the ROI have (H, S, V) values ​​that simultaneously satisfy a preset copper wire color range. Finally, the system compares the total number of copper wire pixels with a preset area threshold (e.g., 10 pixels to filter out noise). If the total number of copper wire pixels within the ROI exceeds this area threshold, the system determines that the area has an exposed copper wire defect and outputs the corresponding defect feature code.

[0048] Example 4: This example illustrates a preferred configuration of the image acquisition module 101. Automotive wiring harness terminals are complex three-dimensional structures, and using a single camera for inspection presents significant visual occlusion problems. For example, a camera viewing from directly above cannot see the state of the crimped seam on the back of the terminal crimping area, while the tightness of the seam is a critical quality indicator.

[0049] To address the occlusion problem in single-point perspective, the image acquisition module 101 in this embodiment includes at least two CCD cameras, which are positioned on different sides of the terminal to form complementary perspectives.

[0050] Specifically, the system includes a front-end CCD camera positioned on one side (e.g., directly above) of the terminal, primarily for observing the open-loop side of the crimp. The camera's field of view is optimized to clearly capture the features to be detected in Embodiments 2 and 3, such as the exposed length of the wire core in the brushed area and whether the wire sheath is pressed into the conductive area.

[0051] The system also includes a rear-end CCD camera positioned on the other side of the terminal (e.g., directly below or to the side). The primary task of this camera is to detect blind spots of the front-end CCD camera. In a preferred configuration, the rear-end camera is specifically used to inspect the condition of the crimped joint on the back of the terminal.

[0052] When working collaboratively, the two (or more) cameras are synchronously triggered via the main controller. When the terminal moves to the precise position of the inspection station, both cameras simultaneously complete exposure, generating two (or more) images. The image processing module 102 loads independent inspection parameters for each of these two images (e.g., loading a dedicated seam inspection tool for the back-end image). During the decision-making process, if a defect is detected in the view of either camera, the system will classify the terminal as a non-conforming product, thus ensuring comprehensive, blind-spot-free inspection of 2D visible features.

[0053] Example 5: This example illustrates an auxiliary module crucial for practical industrial applications: the parameter management module. The automotive wiring harness manufacturing industry typically employs a highly mixed, small-batch production model, where a single production line may need to switch between producing dozens of different product models per day. For each product model, the multimodal system of this invention requires a dedicated and extremely complex testing recipe.

[0054] The complexity of this formulation lies in the fact that it must include parameters for all sub-modules, such as: acquisition parameters for all CCD cameras, ROI coordinates and thresholds for all 2D inspection tools, HSV ranges for all color recognition, configuration parameters for 3D scanners, as well as acceptance criteria for crimp height (CCH) and flare size varying with wire diameter, power and compactness thresholds for ultrasonic testing, and even reconstruction error thresholds and CUSUM control chart parameters used for decision-making in the fusion and prediction module.

[0055] To address the issue of excessively long manual parameter switching times, the parameter management module in this embodiment is configured to enable rapid product model switching through a database-based copy-load mechanism. This module provides a graphical user interface.

[0056] The storage function of this mechanism allows engineers to package the entire set of optimized parameters and store them in the system's recipe database when debugging a new model (such as 305-B) for the first time.

[0057] The mechanism's copy function is used to accelerate the introduction of similar models (such as 305-C). Engineers don't need to start from scratch; they can copy an existing 305-B formula, rename it to 305-C, and then only modify the differing parameters (e.g., adjusting the crimp height tolerance according to standards), and then save. This reduces the introduction time for new products from hours to minutes.

[0058] The loading function of this mechanism enables rapid changeover in production. When an automated production line (e.g., through its MES system) issues a command to load formula 305-C, the parameter management module retrieves the formula file from the database and, within one second, pushes and loads all parameters stored in the formula into all relevant sub-modules (image processing, 3D morphology, ultrasonic compaction, and multimodal data fusion and recognition modules), making the system immediately ready to begin testing new product models.

[0059] Example 6: This example illustrates the working principle of the result display and alarm module. As the system's human-machine interface, its core function is to receive and clearly and operablely display two different types of output signals from the multimodal data fusion and recognition module 105 to the operator and the higher-level system.

[0060] To this end, the module (usually an industrial touchscreen) is configured to provide two distinct feedback modes: bimodal display and tiered alarm.

[0061] The first mode is real-time identification of results, i.e., QC (quality control) alarm for a single component. This mode receives real-time harness quality identification results from the fusion module 105 (e.g., the reconstruction error value calculated for the current component in Example 9). The HMI's main interface instantly displays the component's full-dimensional inspection data, including: a 2D image overlaid with OK / NG markers, a 3D data histogram displaying tolerance compliance (e.g., CCH: 1.61mm (NG)), and an ultrasonic level meter indicating whether it meets the standards. If any dimension of the data is unqualified, resulting in the identification result being judged as defective (NG), the overall status light on the HMI immediately turns red, triggering a high-priority hardware alarm (e.g., a red audible and visual alarm), and simultaneously sending a rejection signal to the downstream actuator.

[0062] The second mode is process status early warning, specifically PdM (predictive maintenance) early warning for the production process. This mode receives non-real-time (time-series analysis-based) process status early warning signals from the fusion module 105 (e.g., the process drift signal emitted by the CUSUM analysis unit in Example 9). The HMI has a separate process status or statistics interface that visually displays the source of the process status early warning signal, such as by plotting the cumulative and control charts as described in Example 9 in real time, allowing the operator to see the historical trend representing the cumulative deviation of the reconfiguration error. As long as the CUSUM curve fluctuates within the control limits, this interface indicates process stability. When the CUSUM analysis unit detects that the curve has exceeded the control limits (e.g., indicating significant wear on the crimping die), this module triggers a different, lower-priority early warning (e.g., a solid yellow indicator light on the workstation is illuminated, and the HMI displays a process drift warning: Please check the crimping die).

[0063] Through this mechanism that separates QC alarms from PdM warnings, this module not only provides immediate evidence for rejecting defective products, but also provides predictive guidance for maintaining equipment to prevent the generation of defective products.

[0064] Example 7: This example is one of the core technical modules of the present invention, and elaborates on the working principle of the three-dimensional topography measurement module 103. In this example, the module is preferably a laser triangulation instrument, also commonly referred to as a laser line scanner.

[0065] The module's operation consists of two parts. The first part is the acquisition of 3D point cloud data, based on optical laser triangulation. A laser source inside the sensor generates a linear laser beam and projects it onto the surface of the object being measured (the terminal crimping area). On the other side of the sensor, a high-speed digital camera is mounted at a known, fixed angle to the laser emitter to observe this laser line projected onto the object's surface. Because the terminal crimping area is a complex, uneven 3D curved surface, the laser line observed by the camera is a distorted curve that shifts according to the object's surface height. The processor inside the sensor uses trigonometric functions to instantaneously calculate the Z-coordinate (i.e., height) of each pixel on this distorted curve. By moving the terminal at a constant speed below the sensor (scanning), the sensor continuously acquires the cross-sectional contour lines at an extremely high frequency (e.g., 3000 times per second). These thousands of contour lines are then combined to generate a set of millions of (X, Y, Z) coordinate points in memory—the 3D point cloud data.

[0066] The second part involves the calculation of 3D geometric parameters, which is achieved through algorithmic calculation of the intermediate product, 3D point cloud data. For the most critical indicator, the crimping height, the algorithm first denoises and aligns the point cloud to a reference plane. Then, it segments the point cloud to represent only the top of the crimping area of ​​the conductive wire core. Next, it searches for and locates the point with the largest Z-coordinate value (or calculates the average value) within this subset. The vertical distance between this Z-coordinate value and the reference plane is calculated as the crimping height. For the flare size, the algorithm segments the region representing the flare and fits a standard geometric model (e.g., a cone) onto this part of the point cloud. Finally, it extracts the parameters (such as the opening diameter) from the successfully fitted model as the flare size.

[0067] Ultimately, the 3D topography measurement module 103 does not output massive point cloud data, but instead sends these calculated, concise values ​​(e.g., CCH = 1.58 mm) as 3D geometric parameters to the fusion module 105 in Example 1.

[0068] Example 8: This example is another core technical module of the present invention, which elaborates on the working principle of the ultrasonic compactness detection module 104. The goal of this module is to detect serious defects hidden inside the metal terminal that cannot be detected by 2D vision and 3D topography measurement, such as internal voids / looseness (air gaps caused by incomplete compaction between wire strands) or missing wires. These defects will drastically increase electrical resistance or reduce mechanical pull-out force.

[0069] The ultrasonic tightness testing module 104 includes a pair of ultrasonic transducers (one transmitter and one receiver), which are preferably integrated into the fixture or anvil of the terminal crimping station (or testing station), precisely aligned, and located on both sides of the terminal crimping area.

[0070] The working principle of this module is based on the acoustic impedance and interface reflection principles in acoustic physics. It infers the internal density of the workpiece by measuring the energy loss (i.e., amplitude attenuation) after the ultrasonic wave penetrates the workpiece. When the terminal is positioned, the transmitting transducer generates a high-frequency (e.g., 5-10MHz) ultrasonic compression wave short pulse. This sound wave is guided towards the terminal crimping area, and the propagation direction is perpendicular to the axial direction of the wire core.

[0071] The propagation of the ultrasonic pulse within the crimped area depends entirely on the density of the interior or the tightness of the mechanical contact. In a high-quality, dense crimp, all wire strands and the inner walls of the terminals are tightly compressed together, eliminating air gaps and creating excellent acoustic coupling. Therefore, the ultrasonic pulse can pass through almost unimpeded, with minimal energy loss and very low amplitude attenuation. Conversely, in a low-quality, loose crimp, the sound wave frequently encounters the metal-air interface during propagation. Due to the extremely large difference in acoustic impedance between metal and air, most of the sound wave energy is reflected and scattered each time it encounters this interface, preventing the sound wave from effectively penetrating the entire crimped area, resulting in very high amplitude attenuation.

[0072] The receiving transducer located on the other side of the terminal is responsible for measuring the residual amplitude of the ultrasonic pulse that successfully penetrated the entire crimping area. The system (specifically executed by the fusion module 105) compares the received pulse amplitude with the original pulse amplitude at the time of transmission to quantify the ultrasonic compression wave amplitude attenuation value (e.g., in decibels (dB)). This attenuation value is strongly correlated with the results of the destructive pull-out force test. If the amplitude attenuation value is very low (i.e., the received amplitude is very high), exceeding a preset threshold, it indicates good internal compactness and is deemed acceptable. If the amplitude attenuation value is very high, below the preset threshold, it indicates the presence of internal looseness or voids and is deemed unacceptable.

[0073] Example 9: As a further refinement of the fusion process described in Example 1, the generation of the multidimensional state vector includes a standardized preprocessing step. Those skilled in the art should understand that the data types from different modules are heterogeneous: two-dimensional defect features (e.g., exposed copper wire, missing sealing ring) from image processing module 102 are discrete categorical data; while 3D geometric parameters (e.g., crimping height 1.58 mm) from three-dimensional topography measurement module 103 and crimping density data (e.g., attenuation value -20 dB) from ultrasonic density detection module 104 are continuous numerical data.

[0074] To fuse these heterogeneous data into a unified vector, the fusion module 105 processes the data as follows: First, for discrete categorical data (two-dimensional defect features), the module employs one-hot encoding. For example, if there are three possible defect types (A, B, C), then A is converted to a vector, B is converted to a vector, and C is converted to a vector.

[0075] Secondly, for continuous numerical data (3D parameters and ultrasonic attenuation values), the module employs normalization to eliminate the influence of different physical dimensions and numerical ranges. A preferred implementation is Z-score normalization, which involves subtracting the mean of that feature from the normal samples from each value and then dividing by its standard deviation; another approach is Min-Max normalization, which linearly scales the values ​​to a fixed interval, such as between 0 and 1.

[0076] Finally, this module concatenates all the numerical vectors processed above (i.e., all one-hot encoded vectors and all normalized continuous values) in a predetermined order. This process concatenates all the data into a single, high-dimensional, flattened feature vector, which becomes the multidimensional state vector input to the subsequent model.

[0077] This embodiment is the brain and the most critical core innovation of the system of the present invention, detailing the internal structure and working principle of the multimodal data fusion and recognition module 105. This module is configured to perform a complex two-stage analysis task: in the first stage, an anomaly detection is performed on the multidimensional state vector of a single terminal using a long short-term memory autoencoder; in the second stage, process drift detection is performed on the time series of reconstruction errors generated in the first stage using a cumulative sum control chart analysis unit.

[0078] The first part is a Long Short-Term Memory (LSTM) autoencoder (for real-time anomaly detection). An autoencoder is an unsupervised neural network whose basic structure consists of a symmetric encoder and decoder. The encoder is responsible for compressing the data, transforming a high-dimensional multidimensional state vector into a low-dimensional latent space representation; the decoder is responsible for reconstructing the data, decompressing the latent representation to reconstruct the original multidimensional state vector. This embodiment uses a more advanced Long Short-Term Memory (LSTM) unit to construct the encoder and decoder because LSTMs are specifically designed for processing sequential data and capturing temporal dependencies, enabling a deeper modeling of dynamic patterns in the wire harness crimping process.

[0079] The core principle of this LSTM autoencoder for anomaly detection lies in its training method. Before deployment, the model is trained using only tens of thousands of multidimensional state vector samples known to be acceptable under normal production conditions (i.e., full-dimensional acceptable data from Examples 2, 7, and 8). The training objective is to minimize the reconstruction error, i.e., the mathematical difference between the original input vector and the reconstructed output vector. Through this training, the model only learns how to efficiently compress and perfectly reconstruct good products; it has never encountered the data patterns of defective products.

[0080] Its working principle is to use reconstruction error for anomaly detection. When a good terminal is detected, its multidimensional state vector is consistent with the normal pattern learned by the model, and the model can reconstruct it very accurately, resulting in a very low reconstruction error. However, when a defective terminal is detected, its vector contains anomalous data patterns that the model has never seen during training, and the model cannot reconstruct it accurately. As a result, there will be a huge mathematical difference between the original input of the defective terminal and the reconstructed (attempted normal) output. This high reconstruction error value is a high-sensitivity anomaly score. The LSTM autoencoder generates a quantized reconstruction error value for each terminal. This value is used to determine whether the terminal is defective in real time, and regardless of whether it is high or low, it will be sent as a data point in real time to the second part of this embodiment.

[0081] The second part is the cumulative sum control chart analysis unit (for process drift detection). The automatic encoder in the first part excels at capturing single, large anomalies, but is not sensitive enough to detect slow process drifts (such as slow wear of a pressing die). For example, die wear might cause the reconstruction error to rise very slightly but steadily from the average of 0.05 to 0.06, a value far below the defect threshold (e.g., 0.8), so the first part won't alarm. The cumulative sum control chart is one of the most sensitive tools in statistical process control for detecting such small deviations or drifts in the process mean. It works by looking at the cumulative deviation rather than the instantaneous value.

[0082] The working principle of this unit is to use the output (reconstruction error) of the first part (automatic encoder) as its input data. The system first determines a target mean value for the reconstruction error (e.g., 0.05). For each new reconstruction error value, this unit calculates its deviation from the target mean (e.g., +0.01) and maintains a continuously accumulating deviation sum (i.e., cumulative sum). When the process is stable, new reconstruction error values ​​fluctuate randomly around the target mean, and the resulting deviations cancel each other out, with the cumulative sum always hovering around zero. However, when the process drifts (e.g., mold wear), the reconstruction error will continuously and systematically exceed the target mean. Although each deviation is extremely small, the CUSUM algorithm accumulates them. When the absolute value of this cumulative sum eventually exceeds the preset control limit, this unit determines that a significant process drift has occurred and immediately generates the process status warning signal described in the example, sending it to Mode 2 of Embodiment 6 to trigger a yellow light warning.

[0083] The synergistic effect of this embodiment lies in the fact that the autoencoder (AE) acts as a powerful feature extractor, transforming high-dimensional, complex multidimensional state vectors into a single, one-dimensional quality indicator (reconstruction error); while the CUSUM analysis unit then performs highly sensitive drift detection on this AE-purified one-dimensional time series data. This combination enables the system to simultaneously capture both sudden major defects and gradual process drift.

[0084] As a further refinement of the autoencoder (AE) described in Embodiment 9, the present invention preferably employs a stacked long short-term memory (LSTM) autoencoder structure. This structure includes an encoder and a decoder.

[0085] The encoder preferably consists of two or more consecutive LSTM layers; for example, the first layer has 64 hidden units, and the second layer has 32 hidden units. The encoder's function is to compress the high-dimensional input vector layer by layer, ultimately outputting a latent space representation vector with significantly reduced dimensionality.

[0086] The decoder has a structure mirror-symmetric to the encoder; for example, the first layer has 32 hidden units, and the second layer has 64 hidden units. The decoder's role is to receive the low-dimensional vector and attempt to decompress it layer by layer, reconstructing it back into an output vector of the same dimension as the original input vector. The internal activation functions of all LSTM units in the network are preferably conventional activation functions in the art, such as tanh (hyperbolic tangent).

[0087] The training objective of this model is to minimize the reconstruction error. Specifically, the preferred loss function is mean squared error, which is calculated as follows: for each dimension of the vector, the square of the difference between the original input value and the reconstructed output value is calculated; then, the average of these squared differences across all dimensions is calculated. The model training process involves adjusting the network parameters to minimize this mean squared error.

[0088] The training process preferably employs an optimizer known in the art, such as the Adam optimizer, and sets an appropriate learning rate (e.g., 0.001) and batch size (e.g., 64). As described in Example 9, the model is trained only on tens of thousands of multidimensional state vectors representing good products, so that it only learns how to accurately reconstruct normal data.

[0089] As a further refinement of the Cumulative and Control Chart (CUSUM) analysis unit described in Example 9, those skilled in the art should understand that the operation of this unit depends on the pre-setting of three key parameters: Target mean (T): The average value of the stable reconstruction error achieved by LSTM-AE on the training set (good dataset).

[0090] Relaxation value (k): Represents the allowable range of normal process fluctuations.

[0091] Control limit (H): Represents the threshold for determining whether a process has runaway.

[0092] The working principle of this unit is that it no longer focuses on whether a single reconstruction error value exceeds the limit, but instead monitors the accumulation of deviations. This unit continuously maintains two accumulated values ​​in memory: an upward accumulated sum and a downward accumulated sum, both initially set to zero.

[0093] For each new reconstruction error value, the cell performs the calculations described in the following logic: First, calculate the upper bound (i.e., the target mean plus the relaxation value) and the lower bound (i.e., the target mean minus the relaxation value).

[0094] Secondly, update the upward cumulative sum: calculate the difference between the current reconstruction error and the upward boundary. If this difference is positive (i.e., the error exceeds the upper boundary), then this positive difference is added to the upward cumulative sum. If this difference is negative (i.e., the error is within the boundary), then it is not added. Furthermore, at any time, if the value of the upward cumulative sum becomes negative after accumulation, the system will force it to be reset to zero (i.e., only the upward deviation is accumulated, and the downward integral is not included).

[0095] Next, update the downward cumulative sum: calculate the difference between the lower boundary and the current reconstruction error. If this difference is positive (i.e., the error is below the lower boundary), add this positive difference to the downward cumulative sum. Similarly, if the value of the downward cumulative sum itself becomes negative, it is also forced to be reset to zero (i.e., only the downward deviation is accumulated).

[0096] Finally, a judgment is made: after each update of the cumulative value, the system checks whether the absolute value of either the upward or downward cumulative sum exceeds the preset control limit (H). If it does, it indicates that a small but continuous drift has occurred in the process mean, and the system immediately generates a process status warning signal.

[0097] Example 10: This example will refer to Figure 2 The flowchart, combined with the modules and principles detailed in the aforementioned embodiments 1 to 9, clarifies the complete execution steps of the CCD vision inspection-based automotive wiring harness quality identification method provided by the present invention.

[0098] This method is applied to the systems described in Examples 1 to 9. When an automotive wiring harness terminal is delivered to the inspection station, the method first performs step (a): by having the image acquisition module (as described in Example 4) and the image processing module (as described in Examples 2 and 3) work together to acquire a 2D image of the terminal and obtain two-dimensional defect features. Specifically, the front-end and back-end CCD cameras of Example 4 acquire 2D images simultaneously. The images are sent to the image processing module of Example 2, which activates its internal set of algorithm tools in parallel, such as activating a material presence detection tool and a wire core exposure detection tool for spot analysis, an edge detection tool for wire insulation crimping, and a color recognition tool based on HSV color space analysis (Example 3) to detect exposed copper wires. The module finally summarizes the results of all tools to generate two-dimensional defect features (e.g., [sealing ring: OK, bristle length: NG]).

[0099] Almost simultaneously with step (a), the method performs step (b): acquiring 3D geometric parameters via a 3D topography measurement module (as described in Example 7). As described in Example 7, a preferred laser triangulation instrument projects a laser line onto the terminal and generates high-density 3D point cloud data using the principle of laser triangulation. The module's internal processor immediately performs algorithmic calculations on the point cloud data, calculating the crimp height (CCH) by searching for the highest point in the Z-coordinate and calculating the flared opening size through geometric fitting. The module ultimately outputs these values ​​(e.g., [CCH: 1.58 mm]) as 3D geometric parameters.

[0100] Almost simultaneously with steps (a) and (b), the method performs step (c): acquiring internal crimp tightness data via an ultrasonic tightness detection module (as described in Example 8). As described in Example 8, an ultrasonic transducer pair integrated on the workstation fixture is activated, with the transmitting end emitting ultrasonic compression wave pulses in the direction of penetrating the crimped area of ​​the terminal. The receiving end measures the residual amplitude of the pulse, and the system quantifies the ultrasonic compression wave amplitude attenuation value accordingly. This attenuation value (a numerical value quantifying internal tightness) is output as crimp tightness data.

[0101] Finally, the method execution step (d) involves fusion and analysis through the multimodal data fusion and recognition module (as described in Example 9). This step first executes sub-step (d1), which involves collecting all heterogeneous output data (two-dimensional features, 3D parameters, and dense data) from steps (a), (b), and (c), preprocessing them (such as normalization), and fusing them into a single, high-dimensional mathematical vector, i.e., a multidimensional state vector, and adding it to the historical time series.

[0102] Next, sub-step (d2) is immediately executed, which generates results and warnings based on time series analysis. This step is executed in parallel by the two-stage analysis unit detailed in Example 9. In Analysis 1 (AE Real-time Anomaly Detection), the multidimensional state vector is fed into a Long Short-Term Memory Autoencoder (LSTM-AE), which attempts to reconstruct the vector and calculates the reconstruction error. If (as in the example of step a) the vector contains NG features, the AE cannot accurately reconstruct it, resulting in a very high reconstruction error that exceeds the preset defect threshold. The system therefore generates a harness quality identification result = defective. In Analysis 2 (CUSUM Process Drift Detection), simultaneously, the calculated (high) reconstruction error value is sent as a new data point to the Cumulative Sum Control Chart Analysis Unit. The CUSUM unit updates its cumulative sum of deviations. If (as described in Example 9) the process is already slowly drifting, this new high error value may cause the cumulative sum to eventually exceed the CUSUM control limit. The system therefore generates an additional process status warning signal = process drift.

[0103] The endpoint of this method is to send the identification result (defective product) and process status warning signal (process drift) generated in sub-step (d2) together to the result display and alarm module (as described in Example 6). The module will then perform a dual alarm: triggering a red QC alarm to remove the defective product, and simultaneously illuminating a yellow PdM warning light to warn the operator to check the process (e.g., check the mold).

[0104] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A CCD vision inspection-based automotive wiring harness quality identification system, characterized in that, The system includes: An image acquisition module configured to acquire 2D images of the automotive wiring harness terminals; An image processing module is connected to the image acquisition module, and the image processing module is configured to identify two-dimensional defect features of the terminal based on the 2D image; A three-dimensional topography measurement module is configured to acquire the 3D geometric parameters of the terminal crimping area; An ultrasonic tightness testing module configured to non-destructively test the crimp tightness inside the terminal; and A multimodal data fusion and recognition module is connected to the image processing module, the three-dimensional topography measurement module, and the ultrasonic compactness detection module, respectively. The multimodal data fusion and recognition module is configured as follows: The two-dimensional defect features, the 3D geometric parameters, and the press-fit compaction are integrated to generate a multi-dimensional state vector; and Based on the time series analysis of the multidimensional state vector, the identification results of the wire harness quality and the early warning signal of the process status are generated. The ultrasonic tightness detection module includes: A pair of ultrasonic transducers integrated into the terminal crimping station; The multimodal data fusion and recognition module is further configured to quantify the press-fit compactness based on the ultrasonic compression wave amplitude attenuation value measured by the ultrasonic transducer. The multimodal data fusion and recognition module includes: A long short-term memory autoencoder; and A cumulative sum and control chart analysis unit; The long short-term memory autoencoder is configured to generate a reconstruction error based on the multidimensional state vector. The cumulative sum and control chart analysis unit is configured to monitor the time series of the reconstruction error to identify process drifts indicating wear of the pressing die and generate the process status warning signal.

2. The system as described in claim 1, characterized in that, The image processing module includes: The material has at least one of the following: a testing tool for the presence or absence of a wire sheath crimping test tool, and a testing tool for the exposure of wire cores on the left and right sides.

3. The system as described in claim 1 or 2, characterized in that, The image processing module further includes: A color recognition tool is configured to detect exposed copper wires at the terminal crimping point by color recognition.

4. The system as described in claim 1, characterized in that, The image acquisition module includes: A front-end CCD camera is disposed on one side of the terminal; and A rear-end CCD camera is located on the other side of the terminal.

5. The system as described in claim 1, characterized in that, The system also includes: A parameter management module is connected to the image processing module, and the parameter management module is configured to enable rapid switching of product models through a copy loading mechanism.

6. The system as described in claim 1, characterized in that, The system also includes: A result display and alarm module, which is connected to the multimodal data fusion and recognition module, is used to receive and display the recognition results and the process status early warning signal.

7. The system as described in claim 1, characterized in that, The three-dimensional topography measurement module includes: A polarization structured light scanner or a laser triangulation instrument; The three-dimensional topography measurement module is configured to: acquire three-dimensional point cloud data of the terminal crimping area, and calculate the 3D geometric parameters based on the three-dimensional point cloud data, wherein the 3D geometric parameters include crimping height and / or flare size.

8. A method for identifying automotive wiring harness quality based on CCD vision inspection, characterized in that, The method is applied to the system as described in any one of claims 1 to 7, and the method includes the following steps: (a) Acquire a 2D image of the terminal using the image acquisition module, and process the 2D image using the image processing module to obtain the two-dimensional defect features; (b) Obtain the 3D geometric parameters of the terminal crimping area using the three-dimensional topography measurement module; (c) Obtaining the crimping tightness data inside the terminal using the ultrasonic tightness testing module; and (d) Through the multimodal data fusion and recognition module: (d1) Integrate the two-dimensional defect features, the 3D geometric parameters, and the press-fit compaction data to generate a multi-dimensional state vector; and (d2) Based on the time series analysis of the multidimensional state vector, generate the identification result of the wire harness quality and the early warning signal of the process status.

Images