A single-terminal state detection method based on a deep detection network

Abstract

The application discloses a single-terminal state detection method based on a deep detection network, and belongs to the technical field of machine vision and industrial defect detection. Non-open source single-terminal images in actual production or detection sites are collected to establish a labeled data set containing normal state items and defect state items; the data set is divided into a training set, a verification set and a test set, and only the training set is subjected to offline enhancement, each training image is randomly selected with 1 to 2 kinds of enhancement modes to generate an enhanced image, and the enhancement modes include blurring, brightness contrast, HSV, gamma, translation scaling rotation and Gaussian noise; a deep detection network containing a configurable improvement unit is constructed, and the target position, category and confidence are output through the main feature extraction, multi-scale feature fusion and detection head. The method can consider normal state recognition and defect detection, support equivalent replacement of light weight or feature enhancement structure, and improve the site detection efficiency and adaptability.

Description

Technical Field

[0001] This invention relates to the fields of machine vision, industrial defect detection, and deep learning target detection, and particularly to a single-terminal state detection method and system based on a deep detection network. Background Technology

[0002] Single-terminal wire harnesses or single-terminal connectors are widely used in electronic assembly, automotive wiring harnesses, industrial control, and consumer electronics. The appearance of a single terminal directly affects conductivity reliability, assembly stability, and product consistency. Therefore, it is necessary to consistently identify its normal and defective states during production or testing.

[0003] In actual production, single-terminal condition inspection typically requires not only determining the presence of defects but also identifying several normal conditions in the terminal structure, such as exposed wire leads, exposed wire cores, exposed insulation, and insulation-pressed feet. Simultaneously, it also requires identifying defective conditions such as exposed wire leads, exposed wire cores, insulation wires, overmolding, and copper plating. These conditions vary considerably in appearance, with wire lead defects being particularly small and irregularly shaped, easily affected by lighting, shooting angle, and batch variations.

[0004] Traditional manual visual inspection methods suffer from low efficiency, inconsistent standards, fatigue, and high false negative rates. Traditional image processing methods typically rely on rules such as threshold segmentation, edge extraction, and morphological analysis, which are insufficiently adaptable to changes in lighting, terminal posture, and complex defect morphologies.

[0005] In recent years, deep learning object detection networks have been used in industrial vision inspection. However, when existing general-purpose detection networks are directly applied to single-terminal condition detection, there are still problems such as limited sample size, missed detection of small defects, high model parameter count and deployment cost, and inconvenient network structure iteration.

[0006] Therefore, there is a need for a detection method and system that is oriented towards the actual acquisition of single-terminal data, can take into account both normal and defective state items, and allows for the configuration of lightweight or feature-enhanced structures according to application needs. Summary of the Invention

[0007] The purpose of this invention is to provide a single-terminal state detection method and system based on deep detection networks, so as to solve the problems of low efficiency of manual detection, insufficient adaptability of traditional image processing, difficulty in identifying small defects, and inconvenience of model structure iteration in existing single-terminal detection.

[0008] To achieve the above objectives, the present invention provides a single-terminal state detection method based on a deep detection network, comprising the following steps:

[0009] S1. Data Acquisition and Labeling Steps: Acquire single-terminal images from actual production or testing sites to establish a single-terminal status image dataset. This dataset is not an open-source dataset but rather image data actually acquired for single-terminal testing tasks. Label the normal and defective status items in the images with bounding boxes and categories.

[0010] S2. Data Partitioning and Offline Augmentation Steps: Divide the single-endpoint state image dataset into training, validation, and test sets. Offline data augmentation is performed only on the training set. For each training image, one or two augmentation methods are randomly selected to generate a corresponding augmented image, ensuring that both the original training image and the augmented image are used during the training phase. The validation and test sets retain the original, realistically captured images.

[0011] The offline data enhancement methods include random blur, random brightness and contrast transformation (bc), random HSV color perturbation (hsv), random gamma transformation (gamma), random translation, scaling and rotation (ssr), and random Gaussian noise (noise).

[0012] S3. Detection Network Construction Steps: Construct a deep detection network for single-terminal state detection. The deep detection network includes a backbone feature extraction module, a multi-scale feature fusion module, and a detection head, used to extract terminal structural features and defect detail features at different scales.

[0013] S4. Configurable Network Improvement Steps: Set up configurable improvement units in the deep detection network. The configurable improvement units are not limited to a single network module; they can be selected from lightweight downsampling units, spatial-channel feature enhancement units, or equivalent structures with the same function, depending on the detection accuracy, number of parameters, inference speed, and deployment platform.

[0014] S5. Model training steps: Train the depth detection network using the original images and offline augmented images in the training set, and select model weights and training parameters using the validation set.

[0015] S6. State Detection Output Step: Input the image of the single terminal to be detected into the trained deep detection network, output the location, category and confidence of the normal state item and the defect state item, and generate a visual detection result.

[0016] Furthermore, the normal condition items include one or more of the following: protruding wire, wire core clipping, exposed wire core, exposed insulation, and insulation clipping; the defect condition items include one or more of the following: protruding wire with loose wire, exposed wire core with loose wire, insulation with loose wire, coated with adhesive, copper plating, and defective.

[0017] Furthermore, the lightweight downsampling unit is positioned at the downsampling location of the backbone feature extraction module and / or the multi-scale feature fusion module to reduce the number of network parameters and computational load. This unit can adopt a structure including pooling branches, convolutional branches, and feature concatenation branches, or it can adopt an equivalent structure with the same downsampling and feature fusion functions.

[0018] Furthermore, the spatial-channel feature enhancement unit is located at at least one scale level of the backbone feature extraction module to enhance the feature representation of small or localized targets such as filament-like, coated, and copper-clad objects. This unit can adopt a structure combining spatial reconstruction and channel reconstruction, or an equivalent structure with the same feature enhancement function.

[0019] Compared with the prior art, the present invention has the following beneficial effects:

[0020] (1) The present invention covers both normal status items and defect status items of a single terminal, and can output specific categories, locations and confidence levels, rather than being limited to simple good / defective product binary classification.

[0021] (2) The dataset of the present invention is collected from actual production or testing sites, which can better reflect the differences in lighting, posture, batch and defect morphology in real terminal images.

[0022] (3) The present invention adopts an offline enhancement strategy for the training set. One or two enhancement methods are randomly selected for each training image to generate a corresponding enhanced image, which can improve the diversity of training samples without polluting the validation set and test set.

[0023] (4) The present invention can be configured with an improved unit that takes into account both lightweight and accuracy requirements. Attached Figure Description

[0024] Figure 1 This is a flowchart of the single-terminal state detection method provided in an embodiment of the present invention;

[0025] Figure 2 This is a flowchart illustrating the dataset construction and offline enhancement process provided in an embodiment of the present invention.

[0026] Figure 3 This is a flowchart of a configurable depth detection network provided in an embodiment of the present invention;

[0027] Figure 4 A system workflow diagram provided for embodiments of the present invention. Detailed Implementation

[0028] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention. For those skilled in the art, equivalent substitutions can be made to the network structure, enhancement parameters, number of categories, and deployment methods without departing from the concept of the present invention.

[0029] Example 1: Construction of a single-terminal state image dataset.

[0030] In this embodiment, single-terminal images are acquired at the actual production or inspection site using an industrial camera or other image acquisition equipment. The acquired objects are single-terminal wire harnesses or single-terminal connectors, and the acquired images include images from different batches, with different terminal orientations, under different lighting conditions, and with different defect morphologies. This dataset is a dataset actually collected for this inspection task and does not rely on open-source general datasets.

[0031] The acquired images are labeled with bounding boxes, and the labeling categories include normal state items and defect state items. Normal state items can include protruding wires, wire core clips, exposed wire cores, exposed insulation, and insulation clips; defect state items can include protruding wires, exposed wire cores, insulation clips, insulation coating, and copper plating. The category set can be added, deleted, or merged according to the specific terminal process requirements.

[0032] Example 2: Offline data augmentation of the training set.

[0033] The labeled single-terminal state image dataset is divided into training, validation, and test sets according to a preset ratio. In one embodiment, a 7:2:1 ratio can be used. After the division, offline augmentation is performed only on the training set. For each original image in the training set, one or two augmentation methods are randomly selected from six options for combination processing to generate an augmented image. Thus, each original image in the training set corresponds to one augmented image, and both the original and augmented images participate in model training.

[0034] The six offline enhancement methods are as follows: blur (random blur, used to simulate slight defocusing); bc (random brightness / contrast transformation, used to simulate differences in light intensity and contrast); hsv (random HSV color perturbation, used to simulate color and exposure fluctuations); gamma (random gamma transformation, used to simulate different brightness responses); SSR (random translation, scaling, and rotation, used to simulate changes in terminal position and orientation); and noise (random Gaussian noise, used to simulate imaging noise). The validation and test sets do not perform these offline enhancements to ensure that the evaluation results reflect the detection performance on real-world acquired images.

[0035] Example 3: Configurable depth detection network.

[0036] like Figure 3As shown, the deep detection network includes a backbone feature extraction module, a multi-scale feature fusion module, and a detection head. The backbone feature extraction module is used to extract multi-level features from a single-end image, the multi-scale feature fusion module is used to fuse shallow detail features and deep semantic features, and the detection head is used to output detection boxes, categories, and confidence scores.

[0037] In one embodiment, the depth detection network can be built based on a single-stage target detection network, such as the YOLO series networks. It should be noted that the core of this invention is not limited to a specific version of the detection network; any network capable of outputting the location, category, and confidence level of normal and defective status items based on a single-terminal status image can be considered an equivalent implementation of this invention.

[0038] In scenarios requiring reduced parameter and computational costs, lightweight downsampling units can be incorporated into the downsampling locations of deep detection networks. These units may include pooling branches, convolutional branches, and feature concatenation branches. In one specific embodiment, ordinary stride convolutional downsampling can be replaced with an ADown-type lightweight downsampling structure. This structure is merely one optional implementation of the lightweight downsampling unit and does not constitute a limitation on the scope of this invention.

[0039] In scenarios requiring improved accuracy in detecting small-scale targets, a spatial-channel feature enhancement unit can be set at at least one scale level in the backbone feature extraction module. This unit may include a spatial reconstruction branch and a channel reconstruction branch. In one specific embodiment, some feature extraction modules in the backbone network can be replaced with a C3-type spatial-channel feature enhancement structure incorporating the SCConv concept. This structure is only an optional implementation of the spatial-channel feature enhancement unit and does not constitute a limitation on the scope of protection of this invention.

[0040] Example 4: Model training and detection output.

[0041] The deep detection network is trained using original images from the training set and offline augmented images. During training, the model learns image features for single-terminal normal and defective states. Model weights are selected using the validation set, and the final detection performance is evaluated using the test set. Evaluation metrics may include accuracy, recall, mAP50, mAP75, mAP50-95, number of parameters, computational cost, and inference time per image.

[0042] When performing inference on the single-ended terminal image to be detected, the trained model outputs the category, bounding box coordinates, and confidence score for each target. The system can filter the detection results based on a preset confidence threshold and remove duplicate detection boxes using non-maximum suppression. Finally, it outputs a visualized annotated image and a detection report.

[0043] like Figure 4As shown, this invention also provides a single-terminal state detection system based on a deep detection network, including an image acquisition module, a data processing module, a model training module, a state detection module, and a result output module. The image acquisition module is used to acquire single-terminal images from the actual field; the data processing module is used to complete dataset construction, annotation management, data partitioning, and offline enhancement of the training set; the model training module is used to construct and train the deep detection network; the state detection module is used to perform inference on the images to be detected; and the result output module is used to output the category, location, confidence level, and visual annotation results.

[0044] This invention is not limited to the embodiments described above. Without departing from the principles of this invention, those skilled in the art can make various improvements and modifications, all of which should be considered within the scope of protection of this invention.

Claims

1. A single-terminal state detection method based on a deep detection network, characterized in that, Includes the following steps: S1. Data acquisition and annotation steps: Acquire single-terminal images from actual production or testing sites, and establish a single-terminal status image dataset. The single-terminal status image dataset includes target boxes and category annotations for normal status items and defective status items. S2. Data partitioning and offline enhancement steps: Divide the single-terminal state image dataset into a training set, a validation set, and a test set. Perform offline data enhancement only on the training set, and randomly select one or two enhancement methods for each original image in the training set to generate a corresponding enhanced image. S3. Detection network construction steps: Construct a deep detection network for single-terminal state detection, the deep detection network including a backbone feature extraction module, a multi-scale feature fusion module and a detection head; S4. Configurable network improvement step: A configurable improvement unit is set in the deep detection network. The configurable improvement unit is used to reduce the number of network parameters and / or enhance the feature representation of small state targets at single terminals. S5. Model training steps: Train the depth detection network using the original image and the corresponding enhanced image, and select model parameters using the validation set; S6. State detection output step: Input the image of the single terminal to be detected into the trained deep detection network, and output the position, category and confidence level of the normal state item and the defect state item.

2. The single-terminal state detection method based on a deep detection network according to claim 1, characterized in that, The single-terminal status image dataset is a non-open-source image dataset acquired by actual acquisition equipment at the production or testing site. The image dataset contains single-terminal images of different batches, different lighting conditions, different terminal postures, and different defect morphologies.

3. The single-terminal state detection method based on a deep detection network according to claim 1, characterized in that, The normal condition items include one or more of the following: protruding wire, wire core pressing foot, exposed wire core, exposed rubber, and rubber pressing foot. The defect condition items include one or more of the following: protruding wire with flying filaments, exposed wire core with flying filaments, rubber with flying filaments, rubber coating, and copper plating.

4. The single-terminal state detection method based on a deep detection network according to claim 1, characterized in that, The offline data enhancement methods include one or more of the following: random blurring, random brightness and contrast transformation, random HSV color perturbation, random gamma transformation, random translation, scaling and rotation, and random Gaussian noise.

5. The single-terminal state detection method based on a deep detection network according to claim 4, characterized in that, The offline data augmentation is performed before training begins, and the validation and test sets are not augmented to maintain the validation and test sets as real captured images.

6. The single-terminal state detection method based on a deep detection network according to claim 1, characterized in that, The configurable improvement unit includes a lightweight downsampling unit, which is set at the downsampling position of the backbone feature extraction module and / or the multi-scale feature fusion module, and is used to reduce the number of network parameters and computational load while maintaining multi-scale feature transfer.

7. The single-terminal state detection method based on a deep detection network according to claim 1, characterized in that, The configurable improvement unit includes a spatial-channel feature enhancement unit, which is set at at least one scale level of the backbone feature extraction module to enhance the feature representation of fine or localized targets such as filament-like, coated, and copper-clad objects.

8. The single-terminal state detection method based on a deep detection network according to claim 1, characterized in that, The detection head is a multi-scale detection head, used to detect single-terminal state targets based on feature maps of different resolutions, and to perform confidence screening and non-maximum suppression processing on the detection boxes.

9. A single-terminal state detection system based on a deep detection network, characterized in that, include: The image acquisition module is used to acquire single-terminal images in actual production or testing sites; The data processing module is used to build a single-terminal state image dataset and perform offline data augmentation on the training set; A model training module for constructing and training the deep detection network according to any one of claims 1 to 8; The status detection module is used to output the location, category, and confidence level of the normal status item and defect status item of a single terminal; the result output module is used to output the detection results and the visualized annotation image.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the single-terminal state detection method based on a deep detection network as described in any one of claims 1 to 8.

Images