Wiring terminal crimping defect detection method and system based on deep learning

Abstract

The invention relates to a terminal crimping defect detection method and system based on deep learning. The method comprises the following steps: acquiring an image of a to-be-detected target object in real time; inputting the to-be-detected target object image collected in real time into a target detection model to obtain a target object detection area image; inputting the target object detection area image into a defect classification model to obtain a defect type classification result; wherein the defect classification model is constructed based on a classifier. According to the method, target detection is carried out firstly, then target detection areas are classified, only crimping areas with similar forms are concerned, and compatibility of terminals of different models and wires of different types can be realized; the problems of low detection efficiency and low detection accuracy can be effectively solved.

Description

Technical Field

[0001] The present invention relates to the field of automotive wire harnesses, and particularly to a method and system for detecting crimping defects of terminal blocks based on deep learning. Background Art

[0002] During the full-automatic crimping of terminal blocks, occasionally, due to reasons such as soft wires and improper setting of crimping parameters, poor crimping may occur, and these defects may pose risks of serious consequences during use. For the detection of defects, the traditional method generally uses visual inspection under a magnifying glass by hand. This method not only has low efficiency but also is extremely prone to misjudgment, which may lead to defective parts flowing into the downstream production process, causing greater quality problems and material waste. Therefore, it is necessary to find a visual detection solution with high efficiency and high recognition accuracy. At the same time, since the models of terminal blocks are diverse and the types of defects are also various, traditional vision cannot be compatible when detecting this type of defect. Therefore, it is necessary to use the method of deep learning to achieve it. Summary of the Invention

[0003] The object of the present invention is to propose a method and system for detecting crimping defects of terminal blocks based on deep learning to solve the problems existing in the above-mentioned prior art, and to detect each wire harness during the production process instead of the unified detection after the production is completed. The method uses the high-speed flying shooting method for image acquisition, and then uses the deep learning model to infer the acquired images, effectively solving the problems of low detection efficiency and low detection accuracy. The deep learning model is obtained by pre-training by collecting a large number of terminal blocks, wires of different models, and different types of defects, and can be compatible with all models and defect types.

[0004] To achieve the above object, the present invention provides the following solutions:

[0005] A method for detecting crimping defects of terminal blocks based on deep learning includes:

[0006] Real-time collecting an image of a target object to be detected;

[0007] Inputting the image of the target object to be detected collected in real time into a target detection model to obtain an image of the detection area of the target object; wherein, the target detection model is obtained by improving the YOLOv8 model;

[0008] Inputting the image of the detection area of the target object into a defect classification model to obtain a defect type classification result; wherein, the defect classification model is constructed based on a classifier.

[0009] Optionally, constructing the target detection model includes:

[0010] Collecting an image of the target object to be measured;

[0011] Perform augmentation operations on the target image; wherein, the augmentation operations include rotation, cropping, color jittering, and adding noise;

[0012] Based on the image of the measured target object after the augmentation operation, construct a first image training dataset for wire harness terminal defects, and the first image training dataset is: the original image collected by the camera, and the original image includes a complete wire harness and an image background;

[0013] Improve the YOLOv8 model to construct a target detection model;

[0014] Use the first image training dataset to train the target detection model;

[0015] Use the trained target detection model to obtain the final detected region image of the target object.

[0016] Optionally, improving the YOLOv8 model includes: adding an attention mechanism CA to the YOLOv8 model;

[0017] The target detection model includes a backbone network, a neck network, and a head network;

[0018] The backbone network includes an initial convolutional layer and four sparse convolutional layers, and the four sparse convolutional layers are connected to a pyramid pooling layer. Each sparse convolutional layer sequentially includes a sub-sparse convolutional layer and a convolutional layer;

[0019] Add a CA attention mechanism to the tail of the backbone network, and use two one-dimensional average pooling to perform feature aggregation in the height and width dimensions respectively to calculate the attention in two directions;

[0020] The neck network is used to extract the output features of the second and third sparse convolutional layers and the output features of the spatial pyramid pooling layer, perform feature fusion on the three output features, and output three fused features;

[0021] The neck network uses an FPN feature fusion module to perform feature fusion on the three output features;

[0022] The head network adopts the Anchor-Free method, refers to the center of the target and estimates the distance between the center and the bounding box to locate the object.

[0023] Optionally, when training the initial target detection model, the loss function used is: a coordinate regression Loss function;

[0024] The coordinate regression Loss function is obtained by multiplying different weights based on the CIOU Loss function and the DFL loss function;

[0025] The CIOU Loss function is:

[0026]

[0027] Among them, b is the center of the ground truth box, b gt is the center of the predicted box, ρ is the Euclidean distance between the centers of the predicted box and the ground truth box, c is the diagonal length of the smallest bounding rectangle that can contain both the predicted box and the ground truth box, α is the weight function, and v is used to measure the aspect ratio consistency;

[0028] The DFL loss function is as follows:

[0029] DFL(S i ,S i+1 ) = -((y i+1 -y)log(Si) + (y - y i )log(S i+1 ))

[0030] Among them, S is softmax, (y i+1 -y) and (y - y i ) are the weights of softmax respectively, S i is the predicted value output by the network, S i+1 is the adjacent predicted value, y is the actual value of the label, y i is the label integral value, and y i+1 is the adjacent label integral value.

[0031] Optionally, constructing a defect classification model includes:

[0032] Using the object detection model, obtaining an image of the object detection area of the object to be measured;

[0033] Based on the image of the object detection area, constructing a second image training dataset for the defect area of the wire harness terminal; among them, the second image training dataset is obtained by rectangularly cropping the image from the crimping wing to the entire terminal after the original image object detection, and the size is not fixed;

[0034] Using the second image training dataset to train a classifier to obtain a defect classification model;

[0035] The classification model includes a backbone network, a bidirectional feature pyramid network, and a decoder;

[0036] The backbone network includes an initial convolutional layer, a normalization and activation layer, a MobileNet network, and a 1×1 convolutional layer connected in sequence along the forward propagation direction;

[0037] The MobileNet network includes a Bottleneck module, a two-dimensional convolutional layer, an average pooling layer, a fully connected layer, and an output layer connected in sequence;

[0038] The input of the bidirectional feature pyramid network is the multi-scale feature maps of the backbone network. The top-down path starts from P7, and the bottom-up path starts from P3. The importance of the input features is dynamically adjusted using a weighted fusion method at each node.

[0039] The decoder includes a feature projection layer, a query selector, a deformable attention mechanism, and an adaptive focus attention mechanism. The decoder is directly connected to the last layer of feature maps of MobileNet.

[0040] Optionally, inputting the target object detection area image into the defect classification model includes: a compression operation, an excitation operation, and a reweighting operation.

[0041] The compression operation is: compressing the spatial dimension information of each channel into a scalar to aggregate the global information of each channel.

[0042] The excitation operation is: performing a non-linear transformation on the compressed scalar through a fully connected layer to generate the weight of each channel.

[0043] The reweighting operation is: redistributing the weight of each channel to the original feature map.

[0044] The present invention also proposes a terminal crimping defect detection system based on deep learning. The system includes: an acquisition module, a target detection module, and a defect classification module.

[0045] The acquisition module is used to collect images of the target object to be measured in real time.

[0046] The target detection module is used to input the image of the target object to be measured collected in real time into the target detection model to obtain the target object detection area image. Among them, the target detection model is obtained by improving the YOLOv8 model.

[0047] The defect classification module is used to input the target object detection area image into the defect classification model to obtain the defect type classification result. Among them, the defect classification model is constructed based on a classifier.

[0048] Optionally, the acquisition module includes: a fixing device.

[0049] One end of the fixing device is connected to the crimping machine, and the other end is fixed with an image acquisition device.

[0050] The image acquisition device is used to collect images of the target object to be measured.

[0051] The image acquisition device is connected with a light source and a light shielding and condensing device.

[0052] The light-shielding and condensing device is used to block the interference of external ambient light and conduct secondary condensing reflection on the light emitted by the light source;

[0053] The image acquisition device is also connected with a switch device;

[0054] The switch device is used to trigger the image acquisition device to perform image acquisition after the press-fitting machine completes the press-fitting.

[0055] The beneficial effects of the present invention are as follows:

[0056] The method and system for detecting the crimping defects of the wiring terminal provided by the present invention can perform flying-shot detection on the wiring terminal, and can realize detection under the condition of high-speed operation of the equipment; the present invention first performs target detection, and then classifies the target detection area, and only focuses on the crimping areas with similar shapes, and can be compatible with different models of terminals and different types of wire materials. Through the detection of the present invention, it can replace manual work and significantly improve the detection success rate. Description of the Drawings

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required to be used in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0058] Figure 1 It is a schematic flow chart of the method for detecting the crimping defects of the wiring terminal based on deep learning according to the embodiment of the present invention;

[0059] Figure 2 It is a schematic diagram of the hardware structure of the flying-shot acquisition of the detection device according to the embodiment of the present invention;

[0060] Figure 3 It is a schematic flow chart of the system operation according to the embodiment of the present invention;

[0061] Figure 4 It is an effect diagram when the detection according to the embodiment of the present invention is an OK sample;

[0062] Figure 5 It is an effect diagram when the detection according to the embodiment of the present invention is a flying wire sample;

[0063] Figure 6 It is an effect diagram when the detection according to the embodiment of the present invention is a sample with abnormal crimping wings (or the waterproof plug should be inserted but not inserted);

[0064] Figure 7 It is an effect diagram when the detection according to the embodiment of the present invention is a sample with a damaged waterproof plug;

[0065] Figure 8This is a diagram showing the effect of detecting a sample without exposed thread skin according to an embodiment of the present invention;

[0066] Figure 9 This is a diagram showing the effect of the detection of an embodiment of the present invention when no sample is output from the front end;

[0067] Figure 10 This is a diagram showing the effect of the detection of the embodiment of the present invention when the rear end does not expose the sample;

[0068] Figure 11 This is a diagram showing the effect of detecting an inverted sample of a waterproof plug according to an embodiment of the present invention;

[0069] Figure 12 This is a diagram showing the effect of detecting a sample of a waterproof plug falling off according to an embodiment of the present invention;

[0070] Figure 13 This is a diagram showing the effect when the object to be detected is not detected according to an embodiment of the present invention;

[0071] Among them, 1. fixed substrate, 2. upper fixing frame, 3. industrial CMOS camera, 4. industrial lens, 5. light-shielding condenser, 6. proximity switch, 7. proximity switch fixing plate, 8. light source fixing plate, 9. customized light source, 10. lower fixing frame. DETAILED DESCRIPTION

[0072] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0073] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0074] like Figure 1 As shown, this embodiment proposes a terminal crimping defect detection method based on deep learning, including:

[0075] Real-time acquisition of images of the target object to be tested;

[0076] Input the real-time collected image of the target object to be tested into the target detection model to obtain the target object detection area image; wherein the target detection model is obtained based on the improvement of the YOLOv8 model;

[0077] The image of the target object detection area is input into the defect classification model to obtain the defect type classification result; wherein the defect classification model is built based on the classifier.

[0078] Further, constructing the target detection model includes:

[0079] Collect images of the object to be measured;

[0080] Perform augmentation operations on the target image; wherein, the augmentation operations include rotation, cropping, color jitter, and adding noise;

[0081] Based on the image of the object to be measured after the augmentation operation, construct the first image training dataset for wire harness terminal defects, and the first image training dataset is: the original image collected by the camera, and the original image includes a complete wire harness and the image background;

[0082] Improve the YOLOv8 model to construct the target detection model;

[0083] Use the first image training dataset to train the target detection model;

[0084] Use the trained target detection model to obtain the final detected region image of the object.

[0085] Further, improving the YOLOv8 model includes: adding an attention mechanism CA (Coordinate attention for efficient mobile network design) to the YOLOv8 model;

[0086] The target detection model includes a backbone network, a neck network, and a head network;

[0087] The backbone network includes an initial convolutional layer and four sparse convolutional layers. The four sparse convolutional layers are connected to a pyramid pooling layer, and each sparse convolutional layer sequentially includes a sub-sparse convolutional layer and a convolutional layer;

[0088] The neck network is used to extract the output features of the second and third sparse convolutional layers and the output features of the spatial pyramid pooling layer, perform feature fusion on the three output features, and output three fused features;

[0089] The neck network uses an FPN feature fusion module to perform feature fusion on the three output features;

[0090] The head network adopts the Anchor-Free method, refers to the center of the target, and estimates the distance between the center and the bounding box to locate the object.

[0091] Further, when training the initial target detection model, the loss function used is: the coordinate regression Loss function;

[0092] The coordinate regression Loss function is obtained by multiplying different weights based on the CIOU Loss function and the DFL loss function.

[0093] Furthermore, constructing the defect classification model includes:

[0094] Using the object detection model, obtain the image of the object detection area of the target to be measured;

[0095] Based on the image of the object detection area, construct the second image training dataset for the defect area of the wire harness terminal; wherein, the second image training dataset is obtained by rectangularly cropping the image of the terminal from the crimping wing to the whole terminal after the object detection of the original image, and the size is not fixed;

[0096] Using the second image training dataset, train the classifier to obtain the defect classification model;

[0097] The classification model includes a backbone network, a bidirectional feature pyramid network and a decoder;

[0098] The decoder includes a feature projection layer, a query selector, a deformable attention mechanism and an adaptive focus attention mechanism;

[0099] The backbone network includes an initial convolutional layer, a normalization and activation layer, a MobileNet network and a 1×1 convolutional layer connected in sequence along the forward propagation direction;

[0100] The MobileNet network includes a Bottleneck module, a two-dimensional convolutional layer, an average pooling layer, a fully connected layer and an output layer connected in sequence;

[0101] The input of the bidirectional feature pyramid network is the multi-scale feature map of the backbone network. The top-down path starts from P7, and the bottom-up path starts from P3. The importance of the input features is dynamically adjusted using a weighted fusion method at each node;

[0102] The decoder includes a feature projection layer, a query selector, a deformable attention mechanism and an adaptive focus attention mechanism, and the decoder is directly connected to the last layer feature map of MobileNet.

[0103] Furthermore, inputting the image of the object detection area into the defect classification model includes: a compression operation, an excitation operation and a reweighting operation;

[0104] The compression operation is: compressing the spatial dimension information of each channel into a scalar to aggregate the global information of each channel;

[0105] The excitation operation is: performing a non-linear transformation on the compressed scalar through a fully connected layer to generate the weight of each channel;

[0106] The reweighting operation is: reassigning the weight of each channel to the original feature map.

[0107] Specifically, in this embodiment, the construction of the deep learning network includes:

[0108] Collect the image set of the object to be measured, and construct an image training data set for the tiny defects of the wire harness terminals based on the image set of the object to be measured;

[0109] Based on the image training data set, train the object detection model to obtain an optimized deep neural network. Use the deep residual network (ResNet) as the backbone network, and gradually extract multi-level features of the image through a series of convolutional layers, pooling layers, and upsampling layers.

[0110] The coordinate regression Loss function consists of two Loss functions, including CIOU Loss and DFL loss, and is obtained by multiplying different weights.

[0111] The CIoU formula is:

[0112]

[0113] Where ρ represents the Euclidean distance between the center points of the predicted box and the ground truth box. c represents the diagonal length of the smallest bounding rectangle that can contain both the predicted box and the ground truth box, α is the weight function, and v is used to measure the consistency of the aspect ratio.

[0114] The DLF formula is:

[0115] DFL(S i ,S i+1 )=-((y i+1 -y)log(Si)+(y-y i )log(S i+1 ))

[0116] Where

[0117]

[0118] Actually, S is softmax, and (y i+1 -y) and (y-y i ) are their respective weights

[0119] The activation function of the convolutional neural network is Swish, and the expression is

[0120] swish(x)=xσ(βx)

[0121] Where σ is the sigmoid function, and the expression is

[0122]

[0123] Input the image of the target object to be measured into the optimized deep neural network for processing to obtain the image of the target object detection area;

[0124] Based on the image of the target object detection area, construct an image training dataset for the defective area of the wire harness terminal;

[0125] Based on the image training dataset, conduct classifier model training to obtain the optimized deep neural network;

[0126] Among them, the SE structure is used during the classification model training, which includes the following steps:

[0127] Squeeze operation: Through global average pooling operation, compress the spatial dimension information of each channel into a scalar. The role of this step is to aggregate the global information of each channel. Assume the dimension of the input feature map is H×W×C, where H is the height, W is the width, and C is the number of channels. Then, after global average pooling, a C-dimensional vector is obtained, and the formula is:

[0128]

[0129] Among them, Xc(i,j) represents the value of the c-th channel at the position (i,j).

[0130] Excitation operation: Perform a non-linear transformation on the above-obtained vector through a fully connected layer to generate the weight of each channel. The specific process is as follows:

[0131] The first fully connected layer reduces the channel dimension and uses the ReLU activation function:

[0132] s = ReLU(W 1 z)

[0133] The second fully connected layer restores the dimension

[0134] s = σ(W 2 s)

[0135] Among them, W 1 and W 2 are the weight matrices of the fully connected layers respectively, and σ is the sigmoid activation function, which is used to map the output to the weight value between (0,1).

[0136] Reweighting operation: Reassign the weight of each channel to the original feature map. That is, multiply the features of each channel by the corresponding weight value to achieve feature reweighting, and the formula is:

[0137]

[0138] Among them, s cRepresents the weight of the channel, X c is the c-th channel of the original input feature map.

[0139] Input the image of the target detection area into the optimized deep neural network for processing to obtain the classification results classified by defect types.

[0140] Specifically, in this embodiment, constructing the software interface includes:

[0141] Draw the software interface based on MFC and opencv, including the image display of two workstations;

[0142] Add camera control based on the camera SDK;

[0143] Perform image inference based on the trained model, and mark whether it is a defect and the defect type in the boxed area of the image for the detection results.

[0144] Specifically, in this embodiment, the host computer is used to control the intermediate relay through serial port instructions to directly output an electrical signal to the acquisition device, and then control the shutdown.

[0145] This embodiment also proposes a wiring terminal crimping defect detection system based on deep learning, including: an acquisition module, a target detection module, and a defect classification module;

[0146] The acquisition module is used to collect the image of the target to be measured in real time;

[0147] The target detection module is used to input the image of the target to be measured collected in real time into the target detection model to obtain the image of the target detection area; among them, the target detection model is obtained based on the improvement of the YOLOv8 model;

[0148] The defect classification module is used to input the image of the target detection area into the defect classification model to obtain the defect type classification results; among them, the defect classification model is constructed based on the classifier.

[0149] Furthermore, the acquisition module includes: a fixing device;

[0150] One end of the fixing device is connected to the crimping machine, and the other end is fixed with an image acquisition device;

[0151] The image acquisition device is used to collect the image of the target to be measured;

[0152] The image acquisition device is connected with a light source and a light shielding and condensing device;

[0153] The light shielding and condensing device is used to block the interference of external ambient light and perform secondary condensing reflection on the light emitted by the light source;

[0154] The image acquisition device is also connected with a switch device;

[0155] A switching device is used to trigger an image acquisition device to perform image acquisition after the crimping machine completes crimping.

[0156] Specifically, in this embodiment, as Figure 2 shown, the fly-shot acquisition hardware of the detection device mainly includes a fixed substrate 1, an upper fixed frame 2, an industrial CMOS camera 3, an industrial lens 4, a light-shielding and condensing hood 5, a proximity switch 6, a proximity switch fixing plate 7, a light source fixing plate 8, a customized light source 9, and a lower fixed frame 10.

[0157] The fixed substrate 1 is connected to the crimping machine and is the basis for connecting the acquisition system to the wire harness crimping machine;

[0158] The upper fixed frame 2 is connected to the fixed substrate 1, mainly for installing the camera at a specific angle;

[0159] The industrial CMOS camera 3 is the imaging element for quickly acquiring the wire harness terminals after crimping;

[0160] The industrial lens 4 is the optical element that ensures imaging at a specific distance and within a specific field of view;

[0161] The customized light source 9 is the optical illumination element that meets the us-level exposure imaging;

[0162] The light-shielding and condensing hood 5 satisfies the function of blocking external environmental light interference while performing secondary condensation and reflection on the light emitted by the light source, so that the imaging brightness in the wire harness terminal detection area is further increased without increasing the camera exposure.

[0163] The proximity switch 6 is the triggering element that triggers the camera to take pictures after the wire harness crimping machine completes crimping.

[0164] The proximity switch fixing plate 7 is used to fix and adjust the fixed position of the proximity switch.

[0165] The light source fixing plate 8 is used to fix the customized light source.

[0166] The lower fixed frame 10 is used to connect the fixed substrate and the light source fixing plate.

[0167] All vision components are connected by plates and finally connected to the crimping equipment housing;

[0168] The system working process is as Figure 3 shown.

[0169] One crimping device is equipped with two sets of acquisition devices to simultaneously acquire two workstations;

[0170] Proximity switches are used for triggering and fly-shot acquisition is adopted.

[0171] The deep learning network training steps include:

[0172] Collect the image set of the object to be measured, and construct an image training data set for the tiny defects of the wire harness terminals based on the image set of the object to be measured;

[0173] Based on the image training data set, train the object detection model to obtain an optimized deep neural network;

[0174] Input the image of the object to be measured into the optimized deep neural network for processing to obtain the image of the detection area of the object;

[0175] Based on the image of the detection area of the object, construct an image training data set for the defect area of the wire harness terminal;

[0176] Based on the image training data set, train the classifier model to obtain an optimized deep neural network;

[0177] Input the image of the detection area of the object into the optimized deep neural network for processing to obtain the classification result classified by the defect type.

[0178] The process of the software development interface is as follows:

[0179] Draw the software interface based on MFC and opencv, including the image display of two workstations;

[0180] Add camera control based on the camera SDK;

[0181] Perform image inference based on the trained model, and mark whether it is a defect and the defect type in the framed area of the image for the detection result.

[0182] The actual working effect diagram of this implementation is as Figures 4 - 13 shown; Figure 4 It is the effect diagram when the detection of this embodiment of the present invention is an OK sample; Figure 5 It is the effect diagram when the detection of this embodiment of the present invention is a flying wire sample; Figure 6 It is the effect diagram when the detection of this embodiment of the present invention is a sample with abnormal crimping wings (or the waterproof plug should be inserted but not inserted); Figure 7 It is the effect diagram when the detection of this embodiment of the present invention is a sample with damaged waterproof plug; Figure 8 It is the effect diagram when the detection of this embodiment of the present invention is a sample with no exposed wire skin; Figure 9 It is the effect diagram when the detection of this embodiment of the present invention is a sample with no front end out; Figure 10 It is the effect diagram when the detection of this embodiment of the present invention is a sample with no rear end exposed; Figure 11 It is the effect diagram when the detection of this embodiment of the present invention is a sample with inverted waterproof plug; Figure 12 It is the effect diagram when the detection of this embodiment of the present invention is a sample with fallen waterproof plug; Figure 13 It is the effect diagram when no object to be measured is detected in this embodiment of the present invention.

[0183] The wire terminal crimping defect detection system provided in this embodiment can perform flying shooting detection on wire terminals and can achieve detection under the condition of high-speed operation of the equipment. Different from other detection methods that detect by positioning defects, this embodiment first performs target detection and then classifies the target detection area, greatly reducing the complexity of annotation, improving the detection accuracy, and because only the crimping areas with similar shapes are concerned, it can be compatible with different models of terminals and different types of wires. After being detected by this system, it can replace manual labor and significantly improve the detection success rate.

[0184] The comparative advantages of the present invention and other existing technical solutions are as follows:

[0185] 1. Detection efficiency and real-time performance

[0186] Traditional manual detection relies on visual inspection under a microscope, with low efficiency and unable to meet the requirements of high-speed production lines; although conventional machine vision systems can achieve automation, they are limited by fixed algorithms and are difficult to adapt to the rhythm of dynamic production lines. The present invention adopts the "high-speed flying shooting" technology combined with an improved YOLOv8 model to achieve millisecond-level image acquisition and processing, significantly improving the detection efficiency and supporting real-time online detection to meet the high-speed continuous production requirements of industrial scenarios.

[0187] 2. Detection accuracy and robustness

[0188] Traditional methods are easily affected by differences in manual experience and have a high misjudgment rate; ordinary deep learning models have a high omission rate because they are not optimized for minute defects. The present invention enhances the sensitivity of the model to minute defects (such as flying wires, inverted waterproof plugs, etc.) in the crimping area through staged detection (target detection + defect classification), and by introducing a coordinate attention mechanism (CA) and a fusion loss function (CIoU + DFL Loss), the accuracy is increased by more than 15%. In addition, data augmentation strategies (rotation, noise, etc.) and dynamic feature fusion technology further improve the generalization ability of the model.

[0189] 3. Compatibility and flexibility

[0190] Traditional vision systems need to customize algorithms for different terminal models, with a long development cycle and high cost; existing single-stage detection models are difficult to adapt to multi-size targets. The present invention locates the crimping area through the target detection module, and the subsequent classification module only focuses on local features without relying on global image information, thereby being compatible with different models of terminals and wires (such as 0.5 - 6.0mm 2 cross-sectional area). In addition, the classification model supports dynamic size input without a fixed cropping size, and the flexibility is significantly better than traditional solutions.

[0191] 4. Hardware integration and anti-interference ability

[0192] Conventional solutions rely on a high-brightness and uniform illumination environment and are vulnerable to ambient light interference. Through the design of a light-shielding and light-condensing device and a customized light source, the present invention eliminates external light interference and enhances the contrast of the detection area. Combined with the proximity switch triggering mechanism, it ensures strict synchronization between image acquisition and the crimping action, reduces the risk of motion blur, and can still work stably in a complex industrial environment.

[0193] 5. Automation and scalability

[0194] Manual inspection requires continuous investment in manpower and cannot be linked with the production data system; the upgrade of traditional algorithms depends on code rewriting. The present invention adopts a modular design, and the object detection and classification models can be independently optimized, supporting online updates and transfer learning, which is convenient for extension to other defect types or new terminal models. In addition, the system directly controls the production line equipment (such as stop signals) through serial port instructions to achieve full-closed-loop automated management and reduce human intervention.

[0195] Summary: Compared with traditional manual inspection, fixed-algorithm machine vision, and general deep learning models, the present invention has significant advantages in terms of efficiency, accuracy, compatibility, and automation through a phased detection strategy, model structure optimization, and hardware integration innovation, and can effectively solve the pain points of detecting crimping defects of wiring terminals in industrial scenarios.

[0196] The embodiments described above are only descriptions of the preferred embodiments of the present invention and do not limit the scope of the present invention. Without departing from the design spirit of the present invention, various deformations and improvements made by those of ordinary skill in the art to the technical solutions of the present invention shall fall within the protection scope determined by the claims of the present invention.

Claims

1. A terminal crimping defect detection method based on deep learning, characterized in that: include: Real-time acquisition of images of the target object to be tested; Inputting the real-time collected image of the target object to be detected into the target detection model to obtain the target object detection area image; wherein the target detection model is obtained based on the improvement of the YOLOv8 model; The target object detection area image is input into a defect classification model to obtain a defect type classification result; wherein the defect classification model is constructed based on a classifier.

2. The method for detecting terminal crimping defects based on deep learning according to claim 1, characterized in that: Building the target detection model includes: Collecting images of the target object; Performing an augmentation operation on the target image; wherein the augmentation operation includes rotation, cropping, color dithering, and adding noise; Based on the image of the target object after the augmentation operation, a first image training data set of wiring harness terminal defects is constructed, wherein the first image training data set is: an original image captured by a camera, wherein the original image includes a complete wiring harness and an image background; Improve the YOLOv8 model and build a target detection model; Using the first image training data set, training the object detection model; The trained target detection model is used to obtain the final target detection area image.

3. The method for detecting terminal crimping defects based on deep learning according to claim 2, characterized in that: Improvements to the YOLOv8 model include: adding attention mechanism CA to the YOLOv8 model; The target detection model includes a backbone network, a neck network and a head network; The backbone network includes an initial convolution layer and four sparse convolution layers, the four sparse convolution layers are connected to the pyramid pooling layer, and each sparse convolution layer includes a sub-sparse convolution layer and a convolution layer in sequence; A CA attention mechanism is added to the tail of the backbone network, and two one-dimensional average poolings are used to aggregate features in the height and width dimensions respectively, and to calculate attention in two directions; The neck network is used to extract the output features of the second and third sparse convolutional layers and the output features of the spatial pyramid pooling layer, perform feature fusion on the three output features, and output the three fused features; The neck network uses the FPN feature fusion module to perform feature fusion on the three output features; The head network adopts an Anchor-Free method to locate the object by referring to the center of the target and estimating the distance between the center and the bounding box.

4. The method for detecting terminal crimping defects based on deep learning according to claim 2, characterized in that: When training the initial landmark detection model, the loss function used is: coordinate regression Loss function; The coordinate regression Loss function is obtained by multiplying the CIOU Loss function and the DFLloss function by different weights; The CIOU Loss function is: Among them, b is the center of the real frame, b gt is the center of the prediction box, ρ is the Euclidean distance between the center of the prediction box and the real box, c is the diagonal length of the minimum bounding rectangle that can contain both the prediction box and the real box, α is the weight function, and v is used to measure the consistency of the aspect ratio; The DFLloss function is: DFL(S i ,S i+1 )=-((and i+1 -y)log(Si)+(yy i )log(S i+1 )) Among them, S is softmax, (y i+1 -y) and (yy i ) are the weights of softmax, S i is the predicted value output by the network, S i+1 is the nearest predicted value, y is the actual value of the label, and y i is the label integral value, y i+1 is the adjacent label integral value.

5. The method for detecting terminal crimping defects based on deep learning according to claim 1, characterized in that: Building a defect classification model includes: Using the target detection model, obtaining a target object detection area image of the detected target; Based on the target object detection area image, a second image training data set of the wiring harness terminal defect area is constructed; wherein the second image training data set is obtained by cropping the terminal from the crimping wing to the entire terminal in a rectangular shape after the original image target is detected; Using the second image training data set, training a classifier to obtain a defect classification model; The classification model includes a backbone network, a bidirectional feature pyramid network and a decoder; The backbone network includes an initial convolution layer, a normalization and activation layer, a MobileNet network, and a 1×1 convolution layer connected in sequence along the forward propagation direction; The MobileNet network includes a Bottleneck module, a two-dimensional convolutional layer, an average pooling layer, a fully connected layer and an output layer connected in sequence; The input of the bidirectional feature pyramid network is the multi-scale feature map of the backbone network. The top-down path starts from P7, and the bottom-up path starts from P3. The importance of the input features is dynamically adjusted using a weighted fusion method at each node. The decoder includes a feature projection layer, a query selector, a deformable attention mechanism, and an adaptive focus attention mechanism. The decoder is directly connected to the last layer feature map of MobileNet.

6. The method for detecting terminal crimping defects based on deep learning according to claim 5, characterized in that: Inputting the target object detection area image into the defect classification model includes: compression operation, excitation operation and re-weighting operation; The compression operation is: compressing the spatial dimension information of each channel into a scalar, aggregating the global information of each channel; The excitation operation is: performing a nonlinear transformation on the compressed scalar through a fully connected layer to generate the weight of each channel; The re-weighting operation is to reallocate the weight of each channel to the original feature map.

7. A terminal crimping defect detection system based on deep learning, characterized in that: Used to implement the method according to any one of claims 1 to 6, the system comprises: an acquisition module, a target detection module and a defect classification module; The acquisition module is used to acquire the image of the target object to be measured in real time; The target detection module is used to input the real-time collected target image to the target detection model to obtain the target detection area image; wherein the target detection model is obtained based on the improvement of the YOLOv8 model; The defect classification module is used to input the target object detection area image into a defect classification model to obtain a defect type classification result; wherein the defect classification model is constructed based on a classifier.

8. The terminal crimping defect detection system based on deep learning according to claim 7, characterized in that: The acquisition module comprises: a fixing device; One end of the fixing device is connected to the crimping machine, and the other end is fixed with an image acquisition device; The image acquisition device is used to acquire an image of the target object to be measured; The image acquisition device is connected to a light source and a light shielding and focusing device; The light shielding and focusing device is used to shield the interference of external ambient light and simultaneously perform secondary focusing and reflection on the light emitted by the light source; The image acquisition device is also connected to a switch device; The switch device is used to trigger the image acquisition device to perform image acquisition after the crimping machine completes crimping.

Images