Wire harness terminal crimping defect visual detection system and method based on multi-scale feature fusion

Abstract

The present invention relates to the technical field of defect detection, and discloses a visual inspection system and method for wire harness terminal crimping defects based on multi-scale feature fusion. Leveraging the redundant and complementary characteristics of multi-scale features in combination with a multi-scale edge verification mechanism, when features at a certain scale fail due to interference from reflection, defocus, and noise, weighted correction can be performed using features at other scales, which effectively reduces the impact of metal surface reflection and crimp wrinkle pseudo-textures on dimension measurement, and significantly improves the repeatability and reliability of inspection results in complex industrial on-site environments. Meanwhile, a multi-task inspection architecture with shared features is adopted, where defect inspection and dimension measurement share the same fused feature map, so that defect classification and localization as well as dimension parameters can be output synchronously in one inference. This not only avoids computing power redundancy caused by repeated calculations of multiple modules, but also realizes spatial alignment and judgment linkage between defects and dimensions, which can meet the takt time requirements of high-speed mass production inspection.

Description

Technical Field

[0001] This invention relates to the technical field of defect detection, specifically to a visual inspection system and method for wire harness terminal crimping defects based on multi-scale feature fusion. Background Technology

[0002] With the rapid development of the automotive and consumer electronics industries, wire harnesses, as core components for power and signal transmission, directly determine the reliability and mechanical strength of electrical connections through their crimping quality. Terminal crimping is a critical process in wire harness production, achieving a tight connection between metal terminals and wire harness cores through mechanical extrusion. During production, various defects are prone to occur, such as loose wires, insulation indentation, dimensional deviations, burrs, and cracks. Currently, the industry is gradually adopting machine vision-based inspection solutions, often using improved target detection networks for defect identification or traditional edge detection for partial dimensional measurement, thus replacing manual visual inspection to a certain extent and improving quality control efficiency. Existing visual inspection solutions for terminal crimping have limitations. The scale differences between the crimping objects and the inspection targets are significant, requiring both millimeter-level coverage of the overall crimping area and the identification and dimensional measurement of micrometer-level defects. Most existing solutions employ a multi-scale structure of unidirectional feature pyramids, which can only achieve multi-scale defect identification and cannot simultaneously support high-precision geometric dimensional measurement. Single-scale visual inspection frameworks have inherent limitations. Low-resolution, large-field-of-view modes cannot capture minute features such as burrs and microcracks, and the measurement accuracy of core parameters such as crimping height is difficult to meet tolerance requirements. High-resolution, small-field-of-view modes lack global spatial constraints and are susceptible to surface reflections from metal surfaces. Light and workpiece pose deviations can cause edge positioning offsets, making it impossible to simultaneously achieve both global coverage and microscopic detection accuracy. Furthermore, interference from factors such as crimped wrinkles, pseudo-textures, and ambient light fluctuations reduces the stability of single-scale detection. Most solutions separate qualitative defect detection and quantitative dimensional measurement into independent algorithm modules, resulting in incompatible feature systems. This not only leads to redundant computing power and long detection cycles but also increases the likelihood of spatial misalignment between defect and dimensional assessments. In the wire harness industry, there are numerous terminal models and a wide range of wire diameters. Adapting existing detection models is costly, and their generalization ability is weak in small-sample scenarios, making it difficult to meet the flexible production needs of multiple varieties and small batches.

[0003] Chinese invention patent application CN120635013A, published on September 12, 2025, discloses a method for detecting crimping defects in connector harnesses based on image recognition. This method involves segmenting the crimping area image, extracting local brightness distribution features, analyzing abnormal clusters, detecting exposed port signals and contact uniformity, and combining area ratio judgment to identify crimping defects and output risk indicators. By employing image segmentation and regional brightness comparison analysis, it refines the ability to identify subtle differences in the internal structure of the crimping area. Combined with a spatial distribution abnormal cluster discrimination mechanism, it achieves hierarchical differentiation of three-dimensional structural deformation, hidden defects, and local anomalies. Multi-stage reflection feature sorting and regional abrupt change detection effectively capture subtle differences in the transition area between exposed metal and the covering material. Pixel-level path tracing and grayscale uniformity comparison reflect the contact distribution and stress consistency of the wire crimping part. However, the above technical solution cannot achieve accurate detection of crimping defects in connector harnesses based on multi-scale fusion. Summary of the Invention

[0004] (a) Technical problems to be solved To address the significant scale differences in wire harness terminal crimping inspection objects across various scales, existing visual inspection solutions require both millimeter-level coverage of the entire crimping area and micrometer-level identification and dimensional measurement of minute defects. Most existing solutions employ a multi-scale structure of a unidirectional feature pyramid, enabling only multi-scale defect identification but failing to simultaneously support high-precision geometric dimensional measurement. Single-scale visual inspection frameworks inherently suffer from inherent contradictions: low-resolution, large-field-of-view modes cannot capture minute features such as burrs and microcracks, and the measurement accuracy of core parameters like crimping height is insufficient to meet tolerance requirements; high-resolution, small-field-of-view modes lack global spatial constraints and are susceptible to edge positioning shifts due to metal surface reflections and workpiece pose deviations, failing to simultaneously achieve global coverage and microscopic inspection accuracy. Furthermore, interference from crimping wrinkles, pseudo-textures, and ambient light fluctuations reduces the stability of single-scale inspection. Most solutions... Defect qualitative detection and quantitative dimensional measurement are separated into independent algorithm modules with incompatible feature systems. This not only leads to computational redundancy and long detection cycles but also easily results in spatial misalignment between defect and dimensional judgments. Furthermore, the wire harness industry suffers from a wide variety of terminal models and wire diameters, making it costly to adapt existing detection models and limiting their generalization ability in small-sample scenarios. This makes it difficult to meet the flexible production needs of multiple varieties and small batches. To address these issues, a bidirectional, multi-scale feature fusion mechanism is used to overcome the inherent contradiction between field of view and detection accuracy in single-scale detection. This mechanism enables micron-level dimensional measurement and minute defect identification while fully covering the crimping area. Multi-scale edge verification is introduced to improve robustness under operating conditions, and a shared feature multi-task architecture enables simultaneous output of defect detection and dimensional measurement. Combined with transfer learning for rapid model adaptation, this effectively improves the accuracy, stability, and flexible adaptability of wire harness terminal crimping quality inspection.

[0005] (II) Technical Solution This invention is achieved through the following technical solution: a visual inspection method for wire harness terminal crimping defects based on multi-scale feature fusion, the method comprising the following steps: S1. Acquire the original visual image of the wire harness terminal crimping area online, perform distortion correction and grayscale normalization preprocessing on the image to obtain the standard image to be inspected; S2. Construct a multi-scale feature extraction network. Input the standard image to be detected into the network and extract feature maps of corresponding scales through different downsampling levels to obtain a multi-scale feature sequence containing micro-fine-grained features, meso-transitional features, and macro-global features. S3. A bidirectional feature fusion mechanism is used to perform cross-scale feature fusion on the multi-scale feature sequence, and the high-level semantic information and the low-level detail information are enhanced layer by layer to generate a global unified feature map that integrates multi-dimensional information. S4. Input the global unified feature map into the multi-task detection branch network and output two types of detection results simultaneously: one is the defect type, location and confidence level of the crimping area, and the other is the geometric measurement parameters of crimping height, crimping width, flared mouth size and exposed core wire length. S5. Based on the preset defect judgment rules and dimensional tolerance thresholds, and combining the defect detection results with geometric measurement parameters, output the pass / fail judgment result of the terminal crimping quality.

[0006] Preferably, step S1 includes the following steps: S11. At the terminal crimping and unloading station, a multi-source lighting scheme combining coaxial light and low-angle diffused light is adopted. A complete original visual image including the conductor crimping area, insulation crimping area, front and rear flared mouths, and terminal body is captured by an area array industrial camera. S12. Based on the pre-calibrated camera intrinsic parameter matrix, radial and tangential distortion correction is performed on the original visual image, and then the image grayscale value is normalized and mapped to eliminate illumination fluctuation interference and obtain a standard image to be detected. The resolution of the acquired raw visual images meets the requirement that the physical size of a single pixel is no greater than 0.01 mm.

[0007] Preferably, step S2 includes the following steps: S21. Construct a multi-scale feature extraction network with residual convolutional network as the backbone. Input the standard image to be detected into the backbone network and perform downsampling 4 times through convolutional layers with stride of 2 to generate 4 sets of feature maps with progressively lower resolution. S22. Divide the feature maps of each level into feature attributes: the highest resolution shallow feature map is a micro-fine-grained feature, used to characterize the fine textures and edges of burrs, micro-cracks, and exposed single copper wires; the lowest resolution deep feature map is a macro-global feature, used to characterize the global structure of the overall deformation of the terminal and the overall offset of the crimping area; the intermediate level feature map is a meso-transitional feature, used to connect micro-details and macro-structures, together forming a multi-scale feature sequence.

[0008] Preferably, step S3 includes the following steps: S31. Perform top-down feature fusion: Starting from the deepest macro features, after bilinear interpolation upsampling to the size of the adjacent middle-layer features, add them element by element with the middle-layer features to pass the global spatial semantic constraints to the shallow layers layer by layer. S32. Perform bottom-up feature fusion: Starting from the shallowest micro-features, after convolution downsampling to the size of the adjacent middle-layer features, add them element by element with the middle-layer features to supplement the detailed edge information layer by layer to the deeper layers; S33. Upsample the feature maps of each level that have completed bidirectional fusion to the same resolution, and stitch them together along the channel dimension to generate a global unified feature map that contains both global spatial reference and microscopic detail information.

[0009] Preferably, step S4 includes the following steps: S41. Input the global unified feature map into the shared feature input terminal of the multi-task detection branch network, and connect it to the defect detection branch and the size measurement branch respectively; S42. The defect detection branch outputs the category, bounding box coordinates and confidence level of each defect in the crimping area through the target detection head. The defect categories are any one or more of the following: loose threads, insulation pressed in, terminal deformation, burrs, cracks, plating peeling, missing flared end, and broken core strands. S43. The dimension measurement branch outputs the key edge coordinates of the crimping area through the key point regression head, and converts them into actual physical dimensions by combining them with the pre-calibrated pixel equivalents to obtain the geometric measurement parameters of crimping height, crimping width, front and rear flared mouth dimensions, and core wire exposed length.

[0010] Preferably, the end-to-end detection model, consisting of a multi-scale feature extraction network, a bidirectional feature fusion mechanism, and a multi-task detection branch network, includes the following steps in its training process: S61. Collect qualified samples and defective samples of various types of terminals, and obtain the corresponding standard images to be tested as training input sets; S62. Label the defect category and bounding box position of the images in the training input set as the first training label for defect detection; use a cross-section analyzer to measure the true value of the key dimensions of the samples and label the coordinates of the corresponding edge key points as the second training label for dimension measurement. S63. Combine the classification loss, location loss and regression loss of defect detection with the dimensional measurement loss to construct a joint loss function; S64. Use the backpropagation algorithm to synchronously update the weights of each layer of the network until the joint loss function converges to below the set threshold, and the final trained detection model is obtained.

[0011] Preferably, a multi-scale edge verification mechanism is introduced in the size measurement process of S4, specifically as follows: When no scale anomalies are detected in edge features at a certain scale due to reflection, defocus, or noise interference, the edge features at each scale are directly fused to output the final edge localization result. When edge features at a certain scale are found to be abnormal due to reflection, defocus, or noise interference, the corresponding edge features at other scales are retrieved for weighted correction. The spatial position of the macro-global features is used as the benchmark constraint, and the detailed information of the micro-fine-grained features is used to supplement the accuracy, outputting a stable edge localization result.

[0012] Preferably, the method further includes a wire harness terminal replacement and adaptation step: After inputting the standard dimensional tolerances and defect judgment thresholds corresponding to the new model wire harness terminals, and collecting a small number of new model wire harness terminal samples for labeling, the weights of the backbone network and feature fusion layer are frozen using transfer learning. Only the top-level parameters of the multi-task detection branch are updated. After fine-tuning, the detection of the new model can be achieved.

[0013] Preferably, step S5 includes the following steps: S51. Retrieve the defect judgment rules and dimensional tolerance thresholds corresponding to the current terminal model, compare the detected defect type and confidence level with the defect judgment rules, and compare the geometric measurement parameters with the dimensional tolerance thresholds. S52. When a fatal defect or a general defect is detected that exceeds the allowable level, or when any geometric measurement parameter is found to be outside the tolerance range, the product is deemed unqualified; when there are no fatal defects, general defects meet the requirements, and all dimensional parameters are within the tolerance range, the product is deemed qualified. S53. Output the pass / fail judgment result and synchronize it to the crimping equipment control system to trigger the rejection action of unqualified workpieces.

[0014] A visual inspection system for wire harness terminal crimping defects based on multi-scale feature fusion is provided to implement the aforementioned visual inspection method for wire harness terminal crimping defects based on multi-scale feature fusion; the system includes: The wire harness terminal crimping image acquisition module is installed at the material output station of the crimping machine. It is equipped with an industrial camera, a combined light source and an optical lens to acquire the original visual image of the wire harness terminal crimping area online and complete the preliminary preprocessing. Wire harness terminal crimping multi-scale feature reasoning module, which has a built-in end-to-end multi-scale fusion detection model, is configured to perform wire harness terminal crimping multi-scale feature extraction, cross-scale feature fusion and multi-task detection reasoning, and synchronously output wire harness terminal crimping defect detection results and geometric measurement parameters; Wire harness terminal crimping quality determination module, configured to output a wire harness terminal crimping quality qualification determination result by synthesizing defect and dimension data according to preset determination rules and tolerance thresholds.

[0015] (3) Beneficial Effects The present invention provides a wire harness terminal crimping defect visual inspection system and method based on multi-scale feature fusion. The following beneficial effects are achieved: 1. A multi-level feature sequence ranging from shallow micro to deep macro is generated through a multi-scale feature extraction network, and then a bidirectional feature fusion mechanism is used to realize interactive enhancement of global semantic information and underlying detail information, which not only ensures complete coverage and spatial reference constraints of all regions including the conductor crimping zone, the insulation crimping zone and the horn mouth, but also retains the detail features of micron-level defects and edges, can stably meet the strict measurement requirements of crimping dimension tolerances, and fundamentally solves the inherent contradiction that a single-scale scheme cannot have both the field of view and the detection accuracy.

[0016] 2. By utilizing the redundant complementary characteristics of multi-scale features and cooperating with the multi-scale edge verification mechanism, when the feature at a certain scale fails due to interference of reflection, defocus and noise, weighted correction can be performed through features at other scales, which effectively reduces the influence of metal surface reflection and crimping wrinkle pseudo-texture on dimension measurement, and significantly improves the repeatability and reliability of detection results in complex industrial site environments. Meanwhile, a multi-task detection architecture with shared features is adopted, in which defect detection and dimension measurement share the same fused feature map, and defect classification and positioning as well as dimension parameters can be synchronously output through one reasoning, which not only avoids the computing power redundancy caused by repeated calculation of multiple modules, but also realizes spatial alignment and determination linkage of defects and dimensions, and can meet the detection beat requirements of high-speed mass production.

[0017] 3. The multi-scale fusion backbone network has strong general feature extraction capability. When replacing the model of a wire harness terminal, a transfer learning method is adopted, weights of the backbone network and the feature fusion layer are frozen, and only top parameters of the multi-task detection branch are updated; adaptation can be completed with only a small number of samples of the new model wire harness terminal, which greatly shortens the model change cycle, does not need to remanufacture detection templates and perform large-scale data annotation, and can efficiently adapt to the flexible production demands of multi-variety and small-batch in the wire harness industry. Brief Description of the Drawings

[0018] Figure 1 is a flowchart of the wire harness terminal crimping defect visual inspection method based on multi-scale feature fusion provided by the present invention; Figure 2This is a schematic diagram of the module of the visual inspection system for wire harness terminal crimping defects based on multi-scale feature fusion provided by the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] An embodiment of the visual inspection system and method for wire harness terminal crimping defects based on multi-scale feature fusion is as follows: Example 1

[0021] Please see Figures 1-2 A visual inspection method for wire harness terminal crimping defects based on multi-scale feature fusion, the method includes the following steps: S1. Acquire the original visual image of the wire harness terminal crimping area online, perform distortion correction and grayscale normalization preprocessing on the image to obtain the standard image to be inspected; S1 includes the following steps: S11. At the terminal crimping and unloading station, a multi-source lighting scheme combining coaxial light and low-angle diffused light is adopted. A complete original visual image including the conductor crimping area, insulation crimping area, front and rear flared mouths, and terminal body is captured by an area array industrial camera. In practical implementation, a 25-megapixel global shutter area array industrial camera paired with dual telecentric lenses is used, covering an object-side field of view of 16mm × 16mm. The physical size of a single pixel is approximately 0.008mm, meeting the resolution requirements for micron-level metrology. The lighting system consists of a coaxial light source and a low-angle ring diffused light source: the coaxial light source is perpendicularly incident to suppress the strong specular reflection on the tin-plated terminal surface and weaken the high-gloss artifacts of the crimping wrinkles; the low-angle ring light source is incident at a 15° angle to the workpiece surface to highlight the stepped features and minor defects such as burrs and protrusions at the crimping edges. The light source brightness can be steplessly adjusted from 0-255 levels via the controller to adapt to terminals with different plating materials and wire diameters.

[0022] When the feeding mechanism moves the crimped terminal to the center of the inspection station, the slotted photoelectric sensor triggers the camera to take a picture. The single frame exposure time is set to 200μs to effectively suppress motion blur caused by feeding jitter and obtain an original color image containing the complete crimped area.

[0023] S12. Based on the pre-calibrated camera intrinsic parameter matrix, radial and tangential distortion correction is performed on the original visual image, and then the image grayscale value is normalized and mapped to eliminate illumination fluctuation interference and obtain a standard image to be detected. The resolution of the acquired raw visual images meets the requirement that the physical size of a single pixel is no greater than 0.01 mm.

[0024] In practice, the camera intrinsic parameter matrix and distortion coefficients are obtained in advance using the Zhang Zhengyou checkerboard calibration method. Radial and tangential distortion corrections are then performed on the original image to eliminate residual optical distortion from the telecentric lens and ensure geometric consistency of pixel size. The corrected image is converted into a single-channel grayscale image, and the Min-Max normalization algorithm is used to linearly map the grayscale values ​​to the 0-255 standard range. This compensates for the overall grayscale shift caused by brightness attenuation of different batches of light sources and fluctuations in ambient light, ultimately outputting a standard image to be detected.

[0025] S2. Construct a multi-scale feature extraction network. Input the standard image to be detected into the network and extract feature maps of the corresponding scales through different downsampling levels to obtain a multi-scale feature sequence containing micro-fine-grained features, meso-transitional features, and macro-global features. S2 includes the following steps: S21. Construct a multi-scale feature extraction network with residual convolutional network as the backbone. Input the standard image to be detected into the backbone network and perform downsampling 4 times through convolutional layers with stride of 2 to generate 4 sets of feature maps with progressively lower resolution. In practice, the multi-scale feature extraction network uses ResNet-18 as its backbone network, and the input image size is uniformly scaled to 2048×2048 pixels. The network performs convolution downsampling with a stride of 2 through residual blocks in four stages, and outputs four sets of feature maps with resolutions of 1024×1024, 512×512, 256×256 and 128×128, respectively, and the number of channels are 64, 128, 256 and 512, respectively, forming a multi-scale feature sequence from fine to coarse.

[0026] S22. Divide the feature maps of each level into feature attributes: the highest resolution shallow feature map is a micro-fine-grained feature, used to characterize the fine textures and edges of burrs, micro-cracks, and exposed single copper wires; the lowest resolution deep feature map is a macro-global feature, used to characterize the global structure of the overall deformation of the terminal and the overall offset of the crimping area; the intermediate level feature map is a meso-transitional feature, used to connect micro-details and macro-structures, together forming a multi-scale feature sequence.

[0027] In practical implementation, the shallow feature with a resolution of 1024×1024 has the smallest receptive field and the highest detail density, corresponding to micro-fine-grained features, and can capture burr edges, micro-crack textures, and the morphology of exposed single copper wires at the 0.01mm level; the deep feature with a resolution of 128×128 has the largest receptive field and the strongest semantics, corresponding to macro-global features, and can characterize global structural information such as the overall outline of the terminal, the positional offset of the crimping area, and the deformation of large-area terminals; the mid-layer features with resolutions of 512×512 and 256×256 are meso-level transitional features, responsible for connecting micro-details and macro-semantics, and conveying feature information in a transitional manner.

[0028] By generating multi-level feature sequences from shallow microscopic to deep macroscopic levels through a multi-scale feature extraction network, and then achieving interactive enhancement of global semantic information and low-level detailed information through a bidirectional feature fusion mechanism, it not only ensures complete coverage and spatial reference constraints of the entire area such as conductor crimping area, insulation crimping area, and flared mouth, but also preserves the detailed features of micron-level defects and edges. It can stably meet the stringent requirements for crimping dimensional tolerance measurement, and fundamentally solves the inherent contradiction between field of view and detection accuracy in single-scale schemes.

[0029] S3. A bidirectional feature fusion mechanism is used to perform cross-scale feature fusion on multi-scale feature sequences, which enhances the interaction between high-level semantic information and low-level detail information layer by layer, and generates a global unified feature map that integrates multi-dimensional information. S3 includes the following steps: S31. Perform top-down feature fusion: Starting from the deepest macro features, after bilinear interpolation upsampling to the size of the adjacent middle-layer features, add them element by element with the middle-layer features to pass the global spatial semantic constraints to the shallow layers layer by layer. In practice, starting from the deepest macroscopic features with a resolution of 128×128, the bilinear interpolation method is used to upsample to a size of 256×256, and then the feature map is fused element by element with the mid-level feature map of the same resolution, thus transferring the global spatial semantic constraints to the mid-level. The fused 256×256 feature map is then upsampled to a size of 512×512 and added element by element with the corresponding mid-level features, completing the second round of top-down semantic transfer, so that the mid- and shallow features obtain global spatial benchmark constraints.

[0030] S32. Perform bottom-up feature fusion: Starting from the shallowest micro-features, after convolution downsampling to the size of the adjacent middle-layer features, add them element by element with the middle-layer features to supplement the detailed edge information layer by layer to the deeper layers; In practice, starting from the shallowest layer of micro-features with a resolution of 1024×1024, a 3×3 convolutional layer with a stride of 2 is used to downsample to a size of 512×512. This is then fused element-wise with a mid-layer feature map of the same resolution, supplementing the mid-layer with fine edges and texture details. The fused 512×512 feature map is then downsampled to a size of 256×256 using a convolution with a stride of 2, and added element-wise with the corresponding mid-layer features to complete the second round of bottom-up detail transfer, enabling the mid-to-deep features to obtain fine edge information.

[0031] S33. Upsample the feature maps of each level that have completed bidirectional fusion to the same resolution, and stitch them together along the channel dimension to generate a global unified feature map that contains both global spatial reference and microscopic detail information.

[0032] In practice, all four-level feature maps fused bidirectionally are upsampled to a uniform resolution of 1024×1024 using bilinear interpolation and then stitched together along the channel dimension to generate a global unified feature map with 960 channels. Each location in this feature map simultaneously incorporates information from all four scales, providing both macroscopic global spatial constraints and preserving micrometer-level edge details. This resolves the inherent conflict between single-scale accuracy and coverage at the feature level.

[0033] S4. Input the global unified feature map into the multi-task detection branch network and output two types of detection results simultaneously: one is the defect type, location and confidence level of the crimping area, and the other is the geometric measurement parameters of crimping height, crimping width, flare size and exposed core wire length. S4 includes the following steps: S41. Input the global unified feature map into the shared feature input terminal of the multi-task detection branch network, and connect it to the defect detection branch and the size measurement branch respectively; In practice, the globally unified feature map serves as the shared input for both branches, eliminating the need for repeated feature extraction and significantly reducing computational redundancy. The two branches can infer in parallel, and a single forward propagation can simultaneously output defect detection results and dimensional measurement results.

[0034] S42. The defect detection branch outputs the category, bounding box coordinates and confidence level of each defect in the crimping area through the target detection head. The defect categories are any one or more of the following: loose threads, insulation pressed in, terminal deformation, burrs, cracks, plating peeling, missing flared end, and broken core strands. In practice, the defect detection branch uses a lightweight Anchor-Free target detection head to output detection results for eight common crimp defects, including defect category number, coordinates of the upper left and lower right corners of the bounding box, and confidence score. A confidence threshold of 0.7 is set; defects with scores higher than 0.7 are marked as valid defects, and their location, category, and confidence score are recorded. When multiple types of defects exist in the same area, all valid defect information is output simultaneously.

[0035] S43. The dimension measurement branch outputs the key edge coordinates of the crimping area through the key point regression head, and converts them into actual physical dimensions by combining them with the pre-calibrated pixel equivalents to obtain the geometric measurement parameters of crimping height, crimping width, front and rear flared mouth dimensions, and core wire exposed length.

[0036] In practice, the dimensional measurement branch uses a thermal regression-based key point detection head to output sub-pixel coordinates of 16 key edge points, including 4 sampling points each at the upper and lower edges of the conductor crimping area, 4 sampling points each at the upper and lower edges of the insulation crimping area, 1 sampling point each at the apex of the front and rear flared ends, and 2 sampling points at the exposed end of the core wire. Based on the pre-calibrated pixel equivalent, the pixel coordinate differences are converted into actual physical dimensions. Local fluctuations are eliminated by averaging multiple points, and finally, 5 core geometric parameters are calculated: crimping height, crimping width, front flared end size, rear flared end size, and core wire exposed length.

[0037] The S4 dimension measurement process introduces a multi-scale edge verification mechanism, specifically: When no scale anomalies are detected in edge features at a certain scale due to reflection, defocus, or noise interference, the edge features at each scale are directly fused to output the final edge localization result. When edge features at a certain scale are found to be abnormal due to reflection, defocus, or noise interference, the corresponding edge features at other scales are retrieved for weighted correction. The spatial position of the macro-global features is used as the benchmark constraint, and the detailed information of the micro-fine-grained features is used to supplement the accuracy, outputting a stable edge localization result.

[0038] A confidence score is pre-set for the edge feature of each scale, and an edge feature at a certain scale is determined to have scale abnormality caused by reflection, defocus or noise interference if its score is lower than a preset threshold. When it is not detected that the edge feature at a certain scale has scale abnormality caused by reflection, defocus or noise interference, the corresponding edge features at the four scales are directly weighted and fused, calculated according to the ratio of a micro weight of 0.4, a meso weight of 0.4 and a macro weight of 0.2, and a final edge positioning result is output; when the edge feature at a certain scale has scale abnormality, the abnormal feature of this scale is eliminated, the corresponding edge features at the remaining scales are retrieved to reassign weights for weighted correction, the spatial position of the macro global feature is taken as a reference constraint, and the detail features of the remaining normal scales are used to supplement the precision, so as to output a stable edge positioning result. This mechanism can effectively cope with working condition interferences such as local reflection overexposure and local defocus, and ensure the repeatability and stability of dimension measurement.

[0039] By virtue of the redundant and complementary characteristics of multi-scale features, in cooperation with the multi-scale edge verification mechanism, when the feature at a certain scale fails due to interference from reflection, defocus and noise, weighted correction can be performed through features at other scales, which effectively reduces the influence of metal surface reflection and crimping wrinkle pseudo-texture on dimension measurement, and significantly improves the repeatability and reliability of detection results in the complex environment of industrial sites. Meanwhile, a multi-task detection architecture with shared features is adopted, in which defect detection and dimension measurement share the same fused feature map, and defect classification and positioning as well as dimension parameters can be synchronously output through one inference, which not only avoids the computing power redundancy caused by repeated calculation of multiple modules, but also realizes the spatial alignment and judgment linkage of defects and dimensions, and can meet the detection beat requirements of high-speed mass production.

[0040] S5. According to preset defect judgment rules and dimension tolerance thresholds, the defect detection result and the geometric measurement parameters are integrated to output the qualification judgment result of the terminal crimping quality.

[0041] S5 includes the following steps: S51. Retrieve the defect judgment rule and the dimension tolerance threshold corresponding to the current terminal model, compare the detected defect type and confidence with the defect judgment rule, and compare the geometric measurement parameters with the dimension tolerance threshold; In specific implementation, the system has a built-in terminal model parameter library, which stores the defect grade division rules and dimension tolerance standards corresponding to each terminal. Before detection, the currently produced terminal model is selected, and the corresponding judgment rules and tolerance thresholds are automatically retrieved.

[0042] S52. When a fatal defect is detected, a general defect exceeds the allowable level, or any geometric measurement parameter exceeds the tolerance range, the product is judged to be unqualified; when there is no fatal defect, the general defect meets the requirements, and all dimension parameters are within the tolerance range, the product is judged to be qualified; In practice, defects are categorized into two levels: critical defects and general defects. Critical defects include terminal deformation, broken core strands, and missing flared ends; their detection immediately results in a rejection. General defects include loose threads, insulation indentation, burrs, cracks, and plating peeling; these are rejected when their size exceeds the corresponding allowable threshold. Any dimensional parameter exceeding the tolerance range results in rejection. Workpieces without critical defects, with all general defects within allowable ranges, and all dimensional parameters meeting tolerance requirements are considered acceptable.

[0043] S53. Output the pass / fail judgment result and synchronize it to the crimping equipment control system to trigger the rejection action of unqualified workpieces.

[0044] In practice, the judgment results are synchronized in real time to the PLC control system of the crimping machine via the industrial I / O interface. When a defective workpiece arrives at the rejection station, a high-speed air blowing mechanism is triggered to discharge it into the waste box, while qualified workpieces flow normally into the next process. At the same time, all inspection data is uploaded to the production management system to generate quality reports such as defect type statistics and dimensional CPK analysis.

[0045] The training process of the end-to-end detection model, which consists of a multi-scale feature extraction network, a bidirectional feature fusion mechanism, and a multi-task detection branch network, includes the following steps: S61. Collect qualified samples and defective samples of various types of terminals, and obtain the corresponding standard images to be tested as training input sets; In practice, production samples of 12 mainstream automotive terminals were collected, including qualified samples and various defective samples, covering samples with different wire diameters, different plating layers, and different crimping processes; all samples underwent the S1 preprocessing process to generate standard images to be tested as training input sets.

[0046] S62. Label the defect category and bounding box position of the images in the training input set as the first training label for defect detection; use a cross-section analyzer to measure the true value of the key dimensions of the samples and label the coordinates of the corresponding edge key points as the second training label for dimension measurement. In practice, three senior quality inspectors annotate the images to identify defects, including the defect categories and bounding boxes. The intersection of the annotations from multiple inspectors is used as the final first training label. At the same time, a high-precision terminal cross-section analyzer is used to measure the true values ​​of the crimping height, width, and other dimensions of each sample, and the coordinates of the corresponding 16 key points are marked as the second training label.

[0047] S63. Combine the classification loss, location loss and regression loss of defect detection with the dimensional measurement loss to construct a joint loss function; In practice, the joint loss function is calculated as follows: Total loss = 0.3 × defect classification loss + 0.3 × defect location loss + 0.4 × size regression loss; where the classification loss uses cross-entropy loss, the location loss uses CIoU loss, and the size regression loss uses SmoothL1 loss.

[0048] S64. Use the backpropagation algorithm to synchronously update the weights of each layer of the network until the joint loss function converges to below the set threshold, and the final trained detection model is obtained.

[0049] In practice, the Adam optimizer is used for training, with an initial learning rate of 0.001, a batch size of 8, and 100 training rounds. When the joint loss on the validation set no longer decreases for 10 consecutive rounds, an early stopping mechanism is triggered, and the optimal weights are saved as the final detection model.

[0050] The method also includes wire harness terminal replacement and adaptation steps: After inputting the standard dimensional tolerances and defect judgment thresholds corresponding to the new model wire harness terminals, and collecting a small number of new model wire harness terminal samples for labeling, the weights of the backbone network and feature fusion layer are frozen using transfer learning. Only the top-level parameters of the multi-task detection branch are updated. After fine-tuning, the detection of the new model can be achieved.

[0051] In practice, the standard dimensional tolerances and defect judgment thresholds corresponding to the new model wire harness terminals are entered into the parameter library. A small number of qualified and defective samples of the new model wire harness terminals are collected and labeled. Using transfer learning, all weights of the backbone network and the bidirectional feature fusion layer are frozen, and only the top fully connected layer and detection head parameters of the multi-task detection branch are unfrozen. The new model samples are used to fine-tune the parameters for several rounds. After verifying that the accuracy meets the standard, the new model detection adaptation can be completed without retraining the entire network.

[0052] The multi-scale fusion backbone network has strong general feature extraction capabilities. When changing the wire harness terminal model, the transfer learning method is adopted to freeze the weights of the backbone network and the feature fusion layer, and only the top-level parameters of the multi-task detection branch are updated. Only a small number of new model wire harness terminal samples are needed to complete the adaptation, which greatly shortens the model changeover cycle. There is no need to remake the detection template and large-scale data annotation, which can efficiently adapt to the flexible production needs of multiple varieties and small batches in the wire harness industry. Example 2

[0053] Please see Figures 1-2 A visual inspection system for wire harness terminal crimping defects based on multi-scale feature fusion is provided to implement a visual inspection method for wire harness terminal crimping defects based on multi-scale feature fusion. The system includes: The wire harness terminal crimping image acquisition module is installed at the material output station of the crimping machine. It is equipped with an industrial camera, a combined light source and an optical lens to acquire the original visual image of the wire harness terminal crimping area online and complete the preliminary preprocessing. The wire harness terminal crimping multi-scale feature reasoning module has a built-in end-to-end multi-scale fusion detection model, which is used to perform multi-scale feature extraction, cross-scale feature fusion and multi-task detection reasoning for wire harness terminal crimping, and simultaneously outputs the detection results of wire harness terminal crimping defects and geometric measurement parameters. The wire harness terminal crimping quality judgment module is used to output the wire harness terminal crimping quality qualification result based on preset judgment rules and tolerance thresholds, and by combining defect and dimensional data.

[0054] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A visual inspection method for wire harness terminal crimping defects based on multi-scale feature fusion, characterized in that, The method includes the following steps: S1. Acquire the original visual image of the wire harness terminal crimping area online, perform distortion correction and grayscale normalization preprocessing on the image to obtain the standard image to be inspected; S2. Construct a multi-scale feature extraction network. Input the standard image to be detected into the network and extract feature maps of corresponding scales through different downsampling levels to obtain a multi-scale feature sequence containing micro-fine-grained features, meso-transitional features, and macro-global features. S3. A bidirectional feature fusion mechanism is used to perform cross-scale feature fusion on the multi-scale feature sequence, and the high-level semantic information and the low-level detail information are enhanced layer by layer to generate a global unified feature map that integrates multi-dimensional information. S4. Input the global unified feature map into the multi-task detection branch network and output two types of detection results simultaneously: one is the defect type, location and confidence level of the crimping area, and the other is the geometric measurement parameters of crimping height, crimping width, flared mouth size and exposed core wire length. S5. Based on the preset defect judgment rules and dimensional tolerance thresholds, and combining the defect detection results with geometric measurement parameters, output the pass / fail judgment result of the terminal crimping quality.

2. The visual inspection method for wire harness terminal crimping defects based on multi-scale feature fusion according to claim 1, characterized in that, S1 includes the following steps: S11. At the terminal crimping and unloading station, a multi-source lighting scheme combining coaxial light and low-angle diffused light is adopted. A complete original visual image including the conductor crimping area, insulation crimping area, front and rear flared mouths, and terminal body is captured by an area array industrial camera. S12. Based on the pre-calibrated camera intrinsic parameter matrix, radial and tangential distortion correction is performed on the original visual image, and then the image grayscale value is normalized and mapped to eliminate illumination fluctuation interference and obtain a standard image to be detected. The resolution of the acquired raw visual image meets the requirement that the physical size corresponding to a single pixel is no greater than 0.01 mm.

3. The visual inspection method for wire harness terminal crimping defects based on multi-scale feature fusion according to claim 1, characterized in that, S2 includes the following steps: S21. Construct a multi-scale feature extraction network with residual convolutional network as the backbone. Input the standard image to be detected into the backbone network and perform downsampling 4 times through convolutional layers with stride of 2 to generate 4 sets of feature maps with progressively lower resolution. S22. Divide the feature maps of each level into feature attributes: the highest resolution shallow feature map is a micro-fine-grained feature, used to characterize the fine textures and edges of burrs, micro-cracks, and exposed single copper wires; the lowest resolution deep feature map is a macro-global feature, used to characterize the global structure of the overall deformation of the terminal and the overall offset of the crimping area; the intermediate level feature map is a meso-transitional feature, used to connect micro-details and macro-structures, together forming a multi-scale feature sequence.

4. The visual inspection method for wire harness terminal crimping defects based on multi-scale feature fusion according to claim 3, characterized in that, S3 includes the following steps: S31. Perform top-down feature fusion: Starting from the deepest macro features, after bilinear interpolation upsampling to the size of the adjacent middle-layer features, add them element by element with the middle-layer features to pass the global spatial semantic constraints to the shallow layers layer by layer. S32. Perform bottom-up feature fusion: Starting from the shallowest micro-features, after convolution downsampling to the size of the adjacent middle-layer features, add them element by element with the middle-layer features to supplement the detailed edge information layer by layer to the deeper layers; S33. Upsample the feature maps of each level that have completed bidirectional fusion to the same resolution, and stitch them together along the channel dimension to generate a global unified feature map that contains both global spatial reference and microscopic detail information.

5. The visual inspection method for wire harness terminal crimping defects based on multi-scale feature fusion according to claim 1, characterized in that, S4 includes the following steps: S41. Input the global unified feature map into the shared feature input terminal of the multi-task detection branch network, and connect it to the defect detection branch and the size measurement branch respectively; S42. The defect detection branch outputs the category, bounding box coordinates and confidence level of each defect in the crimping area through the target detection head. The defect categories are any one or more of the following: loose threads, insulation pressed in, terminal deformation, burrs, cracks, plating peeling, missing flared end, and broken core strands. S43. The dimension measurement branch outputs the key edge coordinates of the crimping area through the key point regression head, and converts them into actual physical dimensions by combining them with the pre-calibrated pixel equivalents to obtain the geometric measurement parameters of crimping height, crimping width, front and rear flared mouth dimensions, and core wire exposed length.

6. The visual inspection method for wire harness terminal crimping defects based on multi-scale feature fusion according to claim 1, characterized in that, The training process of the end-to-end detection model, which consists of a multi-scale feature extraction network, a bidirectional feature fusion mechanism, and a multi-task detection branch network, includes the following steps: S61. Collect qualified samples and defective samples of various types of terminals, and obtain the corresponding standard images to be tested as training input sets; S62. Label the defect category and bounding box position of the images in the training input set as the first training label for defect detection; use a cross-section analyzer to measure the true value of the key dimensions of the samples and label the coordinates of the corresponding edge key points as the second training label for dimension measurement. S63. Combine the classification loss, location loss and regression loss of defect detection with the dimensional measurement loss to construct a joint loss function; S64. Use the backpropagation algorithm to synchronously update the weights of each layer of the network until the joint loss function converges to below the set threshold, and obtain the finally trained detection model.

7. The visual inspection method for wire harness terminal crimping defects based on multi-scale feature fusion according to claim 1, characterized in that, The S4 dimension measurement process introduces a multi-scale edge verification mechanism, specifically: When no scale anomalies are detected in edge features at a certain scale due to reflection, defocus, or noise interference, the edge features at each scale are directly fused to output the final edge localization result. When edge features at a certain scale are found to be abnormal due to reflection, defocus, or noise interference, the corresponding edge features at other scales are retrieved for weighted correction. The spatial position of the macro-global features is used as the benchmark constraint, and the detailed information of the micro-fine-grained features is used to supplement the accuracy, outputting a stable edge localization result.

8. The visual inspection method for wire harness terminal crimping defects based on multi-scale feature fusion according to claim 1, characterized in that, The method also includes a wire harness terminal replacement and adaptation step: After inputting the standard dimensional tolerances and defect judgment thresholds corresponding to the new model wire harness terminals, and collecting a small number of new model wire harness terminal samples for labeling, the weights of the backbone network and feature fusion layer are frozen using transfer learning. Only the top-level parameters of the multi-task detection branch are updated. After fine-tuning, the detection of the new model can be achieved.

9. The visual inspection method for wire harness terminal crimping defects based on multi-scale feature fusion according to claim 1, characterized in that, S5 includes the following steps: S51. Retrieve the defect judgment rules and dimensional tolerance thresholds corresponding to the current terminal model, compare the detected defect type and confidence level with the defect judgment rules, and compare the geometric measurement parameters with the dimensional tolerance thresholds. S52. When a fatal defect or a general defect is detected that exceeds the allowable level, or when any geometric measurement parameter is found to be outside the tolerance range, the product is deemed unqualified; when there are no fatal defects, general defects meet the requirements, and all dimensional parameters are within the tolerance range, the product is deemed qualified. S53. Output the pass / fail judgment result and synchronize it to the crimping equipment control system to trigger the rejection action of unqualified workpieces.

10. A visual inspection system for wire harness terminal crimping defects based on multi-scale feature fusion, used to implement the visual inspection method for wire harness terminal crimping defects based on multi-scale feature fusion as described in any one of claims 1-9; characterized in that, The system includes: The wire harness terminal crimping image acquisition module is installed at the material output station of the crimping machine. It is equipped with an industrial camera, a combined light source and an optical lens to acquire the original visual image of the wire harness terminal crimping area online and complete the preliminary preprocessing. The wire harness terminal crimping multi-scale feature reasoning module has a built-in end-to-end multi-scale fusion detection model, which is used to perform multi-scale feature extraction, cross-scale feature fusion and multi-task detection reasoning for wire harness terminal crimping, and simultaneously outputs the detection results of wire harness terminal crimping defects and geometric measurement parameters. The wire harness terminal crimping quality judgment module is used to output the wire harness terminal crimping quality qualification result based on preset judgment rules and tolerance thresholds, and by combining defect and dimensional data.

Images