Connector defect identification method, system, equipment and medium

Abstract

The invention discloses a connector defect identification method, system and device and a medium. The method comprises the following steps: acquiring a to-be-detected image containing information of an external surface and an internal structure of a connector; inputting a to-be-detected image into the multiple self-supervised feature learning network; processing the to-be-detected image through a characterization disturbance learning branch in the multiple self-supervised feature learning network, and outputting a first intermediate feature map; processing the to-be-detected image through a structure correlation learning branch in the multiple self-supervised feature learning network, and outputting a second intermediate feature map; fusing the first intermediate feature map and the second intermediate feature map to generate a fused depth feature map; and based on the fused depth feature map, performing defect classification and positioning processing, and obtaining and outputting a defect identification result corresponding to the to-be-detected image. According to the invention, the accuracy and stability of connector defect detection are improved, and multi-variety and small-batch flexible production requirements can be met.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and industrial product quality inspection technology, and in particular to a method, system, device and medium for connector defect identification. Background Technology

[0002] With the continuous improvement of industrial automation, connectors, as key components in electronic devices that enable signal transmission and energy supply, directly affect the reliability of the entire system. Currently, connector defect detection mainly relies on manual visual inspection or traditional machine vision technology. Manual inspection is not only inefficient and labor-intensive, but also prone to high false negative rates due to visual fatigue, making it difficult to meet the demands of high-speed production lines. Traditional machine vision methods achieve defect judgment through preset rules and template matching. Although this improves the detection speed, the recognition accuracy drops significantly when faced with complex defects such as tiny scratches on the connector surface, internal hidden cracks, and assembly deviations of structural components. Especially when the defect shape is diverse, the location is random, or the features are not obvious, false positives and false negatives remain prominent. In addition, existing vision inspection systems have difficulty extracting image features when dealing with highly reflective metal surfaces or complex background interference, resulting in insufficient detection stability and difficulty adapting to flexible production modes of multiple varieties and small batches, which seriously restricts the further improvement of product quality control.

[0003] Therefore, there is an urgent need to provide a technical solution to address the above problems. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a connector defect identification method, system, device, and medium.

[0005] In a first aspect, the present invention provides a method for identifying connector defects, the technical solution of which is as follows: Acquire an image of the connector to be inspected, containing information about the connector's external surface and internal structure; The image to be detected is input into a pre-trained multi-self-supervised feature learning network, which is pre-trained using an unlabeled image set containing multiple categories of connectors through a self-supervised learning task. The image to be detected is processed by the representation perturbation learning branch in the multi-self-supervised feature learning network to output a first intermediate feature map. The first intermediate feature map represents a perturbation invariance feature that is robust to surface reflection, imaging noise and small morphological changes. The image to be detected is processed by the structural association learning branch in the multi-self-supervised feature learning network to output a second intermediate feature map, which represents the structural topology and assembly relationship features between different component regions in the image to be detected. The first intermediate feature map and the second intermediate feature map are fused to generate a fused depth feature map; Based on the fused depth feature map, defect classification and localization processing is performed to obtain and output the defect identification result corresponding to the image to be detected. The defect identification result includes the defect category and the location information of the defect in the image.

[0006] The beneficial effects of the connector defect identification method of the present invention are as follows: The method of this invention extracts and fuses perturbation-invariant features and structural correlation features from unlabeled data through a multi-self-supervised feature learning network. This solves the problems of traditional methods in extracting features from highly reflective metal surfaces and complex background interference, as well as the insufficient accuracy in identifying micro-scratches, hidden cracks, and assembly deviations. It improves the accuracy and stability of connector defect detection and adapts to the needs of flexible production of multiple varieties and small batches.

[0007] Based on the above scheme, the connector defect identification method of the present invention can be further improved as follows.

[0008] Furthermore, the step of acquiring the image to be detected, which includes information about the outer surface and internal structure of the connector, includes: The visible light reflection image and near-infrared transmission image of the connector are acquired simultaneously by multispectral imaging, and the visible light reflection image and the near-infrared transmission image are spatially registered and channel stitched to generate the image to be detected.

[0009] Furthermore, the self-supervised learning task includes a comparative learning task based on the decoupling of cross-spectral consistency and spectral specificity.

[0010] Furthermore, the step of fusing the first intermediate feature map and the second intermediate feature map includes: A dynamic gating fusion method based on channel information entropy and spatial structure saliency is adopted.

[0011] Furthermore, the step of performing defect classification and location processing includes: A detection head network based on deformable convolution and multi-scale feature pyramids is used to process the fused depth feature map.

[0012] Furthermore, the method also includes: After the multi-self-supervised feature learning network is pre-trained, the network parameters are adjusted in a targeted manner using limited labeled samples of the target new category connector based on a few-shot learning strategy.

[0013] Furthermore, the dynamic gating fusion method based on channel information entropy and spatial structure saliency generates the fused deep feature map by jointly optimizing the channel weight vector and the spatial attention mask; The formula for calculating the fused depth feature map is as follows: In the formula, This represents the first intermediate feature map. This represents the second intermediate feature map. This represents the fused depth feature map; This represents the channel weight vector, whose dimension is equal to the number of channels in the feature map. The calculation method is the same: This represents the calculation of the information entropy of the input feature map in each channel. This represents a vector concatenation operation. This is the scaling factor; The spatial attention mask matrix is ​​calculated as follows: This indicates an operation to extract the spatial gradient magnitude from the input feature map. This indicates element-wise multiplication; This represents the operation of converting a vector into a diagonal matrix.

[0014] Secondly, the present invention provides a connector defect identification system, the technical solution of which is as follows: The acquisition module is used to acquire an image to be inspected that contains information about the external surface and internal structure of the connector; The input module is used to input the image to be detected into a pre-trained multi-self-supervised feature learning network, which is pre-trained using an unlabeled image set containing multiple category connectors through a self-supervised learning task. The extraction module is used to process the image to be detected through the representation perturbation learning branch in the multiple self-supervised feature learning network and output a first intermediate feature map. The first intermediate feature map represents perturbation invariance features that are robust to surface reflection, imaging noise and small morphological changes. The detection module is used to process the image to be detected through the structural association learning branch in the multiple self-supervised feature learning network and output a second intermediate feature map, which represents the structural topology and assembly relationship features between different component regions in the image to be detected. The fusion module is used to fuse the first intermediate feature map and the second intermediate feature map to generate a fused depth feature map; The identification module is used to perform defect classification and localization processing based on the fused depth feature map, and to obtain and output the defect identification result corresponding to the image to be detected. The defect identification result includes the defect category and the location information of the defect in the image.

[0015] The beneficial effects of the connector defect identification system of the present invention are as follows: The system of this invention extracts and fuses perturbation-invariant features and structural correlation features from unlabeled data through a multi-self-supervised feature learning network. This solves the problems of traditional methods, such as difficulty in feature extraction on highly reflective metal surfaces and complex background interference, as well as insufficient accuracy in identifying micro-scratches, hidden cracks, and assembly deviations. It improves the accuracy and stability of connector defect detection and adapts to the needs of flexible production of multiple varieties and small batches.

[0016] Thirdly, the technical solution of an electronic device according to the present invention is as follows: It includes a memory, a processor, and a program stored in the memory and running on the processor, wherein the processor executes the program to implement the steps of the connector defect identification method of the present invention.

[0017] Fourthly, the technical solution of a computer-readable storage medium provided by the present invention is as follows: The computer-readable storage medium stores instructions that, when read, cause the computer-readable storage medium to perform the steps of the connector defect identification method of the present invention.

[0018] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0019] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a flowchart illustrating an embodiment of a connector defect identification method according to the present invention; Figure 2 This is a schematic diagram of an embodiment of a connector defect identification system according to the present invention; Figure 3 This is a schematic diagram of an embodiment of an electronic device according to the present invention. Detailed Implementation

[0020] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0021] Figure 1 A flowchart illustrating an embodiment of a connector defect identification method provided by the present invention is shown. Figure 1 As shown, it includes the following steps: S1. Obtain the image to be inspected, which contains information about the external surface and internal structure of the connector.

[0022] In this context, a connector refers to a key interface component in an electronic device used for signal transmission and power supply, typically comprising metal pins, an insulating shell, and other components. For example, a Type-C connector has a plastic shell and contains two rows of 24 metal pins for transmitting current and data signals between the phone and the charging cable. External surface and internal structure information refers to the visible physical form, texture, and color of the connector's exterior, as well as the spatial arrangement and material continuity of its internal components. For example, for a Type-C connector, external surface information includes scratches or dents on the metal interface, while internal structure information includes whether the pins are neatly arranged within the plastic base or whether internal cracks exist. The image to be inspected refers to a digital image acquired by an imaging device that contains information about the external surface and internal structure of the connector to be evaluated. For example, a color image with a resolution of 2048x1536 pixels acquired by an imaging system clearly shows the external outline, surface texture, and internal pin array of the Type-C connector after penetrating imaging.

[0023] S2. Input the image to be detected into a pre-trained multi-self-supervised feature learning network, which is pre-trained using an unlabeled image set containing multiple categories of connectors through a self-supervised learning task.

[0024] Among them, "multiple self-supervised feature learning network" refers to a deep learning network architecture pre-trained through a self-supervised learning task. The network contains multiple parallel feature learning branches used to extract effective feature representations from different aspects of unlabeled data. For example, a neural network model containing two parallel sub-networks is trained on a large number of unlabeled connector images to learn how to extract general features useful for defect detection from the images. "Multi-category connectors" refers to different types of connectors that differ in shape, size, number of pins, or function. For example, categories include connectors with various physical forms, such as Type-C, HDMI, USB-A, and rectangular aviation plugs. "Unlabeled image set" refers to an image set that does not contain any manually labeled information regarding the presence, location, or category of defects. For example, a database containing 100,000 images of various connectors taken under different lighting and angles, used only as training data and not manually labeled as "qualified" or "scratched." Self-supervised learning tasks refer to a learning paradigm that uses the input data itself to construct supervisory signals to train the model, without the need for external annotations. For example, a connector image is randomly rotated by 90 degrees, and then the network is trained to predict the angle of rotation. This task drives the network to learn meaningful features in the image.

[0025] S3. The image to be detected is processed by the representation perturbation learning branch in the multi-self-supervised feature learning network to output a first intermediate feature map. The first intermediate feature map represents a perturbation invariance feature that is robust to surface reflection, imaging noise and small morphological changes.

[0026] The representation perturbation learning branch refers to a sub-network within the multi-layer self-supervised feature learning network specifically designed to learn feature representations that remain invariant to various perturbations in the image. For example, one branch in the network might be trained to ensure that the extracted features from the same connector image are as similar as possible after transformations such as brightness adjustment, noise addition, and minor rotation. The first intermediate feature map refers to the multi-dimensional data matrix output by the representation perturbation learning branch after processing the input image; this matrix represents the features of the network's intermediate layers. For example, a tensor of size 64×64×256, where 64×64 represents the spatial dimension and 256 represents the number of feature channels, encodes features in the input image that are insensitive to changes in illumination. Perturbation-invariant features refer to feature representations that remain stable to photometric changes, geometric deformations, or noise interference in the image. For example, regardless of whether there are highlight reflections on the metal surface of the Type-C connector or Gaussian noise in the image, the core feature vectors extracted by the network regarding the pin edge shape remain essentially consistent.

[0027] S4. The image to be detected is processed by the structural association learning branch in the multi-self-supervised feature learning network to output a second intermediate feature map. The second intermediate feature map represents the structural topology and assembly relationship features between different component regions in the image to be detected.

[0028] The structural association learning branch refers to a sub-network within the multi-self-supervised feature learning network specifically designed to learn the spatial and logical association features between different components in an image; for example, another parallel branch in the network, whose training objective is to infer the relative positional relationships between pins and the casing, and between different pins, from the image. The second intermediate feature map refers to a multi-dimensional data matrix output by the structural association learning branch after processing the input image; for example, a tensor of size 64×64×256, which encodes the spatial relationships and dependencies between various components in the image. Structural topology and assembly relationship features refer to feature representations describing the relative positions, connection methods, and assembly order between multiple components in the image; for example, a feature vector representing spatial constraints such as "the horizontal spacing between pin 3 and pin 5 is 0.5 mm" and "the top of all pins should be aligned with the inner edge of the casing."

[0029] S5. The first intermediate feature map and the second intermediate feature map are fused to generate a fused depth feature map.

[0030] The fused deep feature map refers to a new feature map generated by combining the first intermediate feature map and the second intermediate feature map in a specific way. The new feature map integrates feature information from multiple aspects. For example, two 64×64×256 feature maps are weighted and added together to generate a brand new 64x64x256 feature map. The new map contains both stable surface features and accurate structural relationship features.

[0031] S6. Based on the fused depth feature map, perform defect classification and localization processing to obtain and output the defect identification result corresponding to the image to be detected. The defect identification result includes the defect category and the location information of the defect in the image.

[0032] The defect classification and localization process refers to the calculation process of analyzing the fused depth feature map to determine the type of defect and its location in the original image. For example, analyzing the fused feature map, the output might be "scratches present," with a bounding box coordinate (120, 300, 150, 320), indicating that the scratch is located within that rectangular area in the image. The defect identification result refers to the result data containing specific defect information generated after the defect classification and localization process. For example, a structured output might show "Defect category: bent pin; Location box: top left corner (200, 400), bottom right corner (220, 415)." The defect category refers to the classification of non-conformities in the connector according to their nature; specific categories include "surface scratches," "hidden cracks," "bent pins," and "assembly misalignment." The localization information refers to the spatial location information of the defect area in the image to be inspected.

[0033] The technical solution of this embodiment extracts perturbation invariance features and structural correlation features from unlabeled data through a multi-self-supervised feature learning network and then fuses them. This solves the problems of traditional methods, such as the difficulty in feature extraction on highly reflective metal surfaces and complex background interference, as well as the insufficient accuracy in identifying micro scratches, hidden cracks, and assembly deviations. It improves the accuracy and stability of connector defect detection and adapts to the needs of flexible production of multiple varieties and small batches.

[0034] In one alternative approach, the step of acquiring the image to be detected, which includes information about the connector's outer surface and internal structure, includes: The visible light reflection image and near-infrared transmission image of the connector are acquired simultaneously by multispectral imaging, and the visible light reflection image and the near-infrared transmission image are spatially registered and channel stitched to generate the image to be detected.

[0035] Multispectral imaging refers to an imaging technique that can simultaneously acquire images of the same object at multiple different specific wavelengths. For example, using a special industrial camera, two images of a Type-C connector can be generated simultaneously in the visible light and near-infrared bands in a single shot. Visible light reflection images are images formed by recording the intensity distribution of reflected light from an object's surface in the visible light band. For example, a regular color photograph shows the silver sheen of the metal parts of the Type-C connector and the black texture of its plastic shell. Near-infrared transmission images are images formed by utilizing the penetrating properties of infrared light in the near-infrared band, primarily recording information about the object's internal structure. For example, a grayscale image shows the connector's plastic shell as semi-transparent, clearly revealing the complete shape and arrangement of the internal metal pins. Spatial registration and channel stitching refers to the process of geometrically aligning images from different bands or sources and then combining the image data layers into a multi-channel image file. For example, first, the visible light reflectance image and the near-infrared transmission image are rotated and translated to make the same connector in the image completely aligned. Then, the visible light image is used as the three RGB channels, and the near-infrared image is used as the fourth channel, and they are merged into a four-channel image file.

[0036] In the above-mentioned optional methods, visible light reflection images and near-infrared transmission images are acquired simultaneously through multispectral imaging. The images to be detected are generated through spatial registration and channel stitching, so that a single image can simultaneously carry the external surface texture and internal structural information of the connector, thereby improving the ability to detect surface defects and internal hidden cracks simultaneously.

[0037] In one alternative approach, the self-supervised learning task includes a contrastive learning task based on the decoupling of cross-spectral consistency and spectral specificity.

[0038] Among them, the contrastive learning task based on the decoupling of cross-spectral consistency and spectral specificity refers to a self-supervised learning task that aims to enable the network to learn the common semantic features between different spectral images, while also being able to distinguish the unique physical features of different bands. For example, training the network to determine whether a pair of visible light image patches and near-infrared image patches come from the same location of the same connector, while another auxiliary task requires the network to identify whether a given feature mainly comes from the visible light or near-infrared image.

[0039] Among the above-mentioned optional approaches, a contrastive learning task is further constructed based on the decoupling of cross-spectral consistency and spectral specificity to guide the network to learn the shared semantic representation between visible light and near-infrared modes, while retaining the unique information of each spectral band, thereby enhancing the alignment accuracy of multispectral features and modal complementarity.

[0040] In one alternative approach, the step of fusing the first intermediate feature map with the second intermediate feature map includes: A dynamic gating fusion method based on channel information entropy and spatial structure saliency is adopted.

[0041] Among them, the channel information entropy and spatial structure saliency refers to an index and method for measuring the richness of information contained in different channels of a feature map and the importance of different spatial positions. For example, calculating the information entropy of each channel in the first intermediate feature map reveals that the 128th channel has the highest entropy value and contains the most information. Simultaneously, calculating the structural gradient at each spatial position of the feature map reveals that the gradient magnitude in the pin tip region is significant, indicating high spatial saliency. The dynamic gating fusion method refers to a feature fusion method that dynamically generates weights based on the content of the input feature map and uses these weights to control the fusion ratio of different feature maps. For example, based on the information entropy of each channel in the first and second intermediate feature maps, a set of 256-dimensional fusion weight vectors is calculated in real time. These weight vectors are used to adjust the contribution ratio of each channel of the two feature maps in the final output.

[0042] In the above-mentioned optional methods, a dynamic gating fusion mechanism is further constructed by using channel information entropy and spatial structure saliency. Based on the information content and spatial gradient saliency of different feature channels, the fusion weights of the features representing the perturbation learning branch and the structural association learning branch are adaptively allocated to improve the discriminative power of the fused deep feature map.

[0043] In one alternative approach, the step of performing defect classification and location processing includes: A detection head network based on deformable convolution and multi-scale feature pyramids is used to process the fused depth feature map.

[0044] Among them, "based on deformable convolution and multi-scale feature pyramid" refers to a network structure that incorporates deformable convolution operations and fuses feature maps of different scales. For example, a detection head network uses deformable convolution kernels to adapt to irregular shapes such as bent pins and fuses features from deep and shallow layers of the network to simultaneously detect large-scale assembly misalignments and small-scale surface scratches. A detection head network refers to a network module attached to the back end of a feature extraction network, specifically designed to output results for a specific task. For example, a lightweight fully convolutional network submodule takes the fused depth feature map as input and outputs a series of prediction results including class confidence and bounding box coordinates.

[0045] Among the above-mentioned optional methods, deformable convolution and multi-scale feature pyramids are further introduced to construct the detection head network, which enables the network to flexibly adapt to the geometric deformation of defects and capture cross-scale features from tiny scratches to large-area assembly deviations at multiple receptive field levels, thereby enhancing the localization and classification performance of defects of different sizes.

[0046] In an alternative approach, the method further includes: After the multi-self-supervised feature learning network is pre-trained, the network parameters are adjusted in a targeted manner using limited labeled samples of the target new category connector based on a few-shot learning strategy.

[0047] Among these, few-shot learning strategy refers to a machine learning method that aims to enable the model to effectively learn new categories or tasks using a very small number of labeled samples. For example, when detecting a new type of connector, only 5 defective and 5 undefective labeled images of that model are used, and the network parameters are quickly adjusted through a specific algorithm to adapt the model to the new model. The target new-category connector refers to a new model or type of connector that needs to be detected, whose data is not included in the model's original training set; for example, a square multi-pin connector used in special equipment that has never been seen before. Limited labeled samples refer to a number of labeled data samples that are far fewer than those required for regular model training; for example, only about 10 to 20 images of the target new-category connector that have been manually labeled with defect locations and types. Targeted adjustment refers to targeted and limited updates to some or all of the model's parameters based on specific target data; for example, keeping most of the backbone parameters of the multi-self-supervised feature learning network unchanged, and only updating the parameters of the last few layers of the detection head network using limited labeled samples through several rounds of gradient descent.

[0048] In the above-mentioned optional methods, after pre-training is completed, the network parameters are adjusted in a targeted manner using limited labeled samples of new type connectors based on a few-shot learning strategy, so that the model can quickly adapt to the defect patterns of new type and new material connectors, thereby reducing the data labeling cost and training time when introducing new types.

[0049] In an alternative approach, the dynamic gating fusion method based on channel information entropy and spatial structure saliency generates the fused deep feature map by jointly optimizing the channel weight vector and the spatial attention mask.

[0050] In this context, the channel weight vector refers to a vector calculated in the dynamic gating fusion method to weight different feature channels. The vector aims to maximize the amount of feature information after fusion. For example, a 256-dimensional weight vector obtained by calculating the entropy of each channel in the first and second intermediate feature maps and then normalizing it using Softmax. Channels with higher entropy values ​​receive greater weight. Spatial attention mask refers to a two-dimensional matrix with the same spatial size as the feature map, where the value at each position represents the importance weight of that spatial position during fusion; for example, a 64x64 matrix M_s, where the values ​​of pin edges and suspected crack areas are close to 1, while the values ​​of uniform background areas are close to 0, is used to highlight important regions spatially.

[0051] The formula for calculating the fused depth feature map is as follows: In the formula, This represents the first intermediate feature map. This represents the second intermediate feature map. This represents the fused depth feature map; This represents the channel weight vector, whose dimension is equal to the number of channels in the feature map. The calculation method is the same: This represents the calculation of the information entropy of the input feature map in each channel. This represents a vector concatenation operation. This is the scaling factor; The spatial attention mask matrix is ​​calculated as follows: This indicates an operation to extract the spatial gradient magnitude from the input feature map. This indicates element-wise multiplication; This represents the operation of converting a vector into a diagonal matrix.

[0052] It should be noted that the formula in this embodiment achieves adaptive feature fusion through two parallel weighting mechanisms: the channel weight vector is calculated based on the information entropy of each feature channel, with channels having higher information entropy values ​​considered to contain richer information and therefore receiving higher weights in the fusion; the spatial attention mask is obtained by calculating and interacting the spatial gradient magnitudes of each feature map. Regions with significant gradients typically correspond to structural changes such as edges and textures in the image, which are more critical for defect localization and thus receive higher attention in the spatial dimension. The formula in this embodiment organically integrates the first intermediate feature map from the representation perturbation learning branch and the second intermediate feature map from the structural association learning branch. It adjusts the global importance of each channel of the two feature maps through the channel weight vector and simultaneously applies local saliency weights to different spatial locations of the feature maps through the spatial attention mask. Finally, it generates a fused deep feature map that is both resistant to imaging perturbations and enhances key structural information, providing a robust and discriminative feature representation for subsequent high-precision defect identification.

[0053] In the above-mentioned optional methods, the uncertainty of each channel feature is further quantified by calculating the channel information entropy, the spatial gradient magnitude is extracted to highlight the significant regions of the structural edge, and the channel weight vector and spatial attention mask matrix are jointly optimized to achieve mathematical interpretability and adaptive adjustment of the feature fusion process.

[0054] To better illustrate the technical solution of this embodiment, the following example is used for complete explanation: S10. Using a multispectral imaging device equipped with visible light and near-infrared sensors, simultaneously acquire visible light reflection and near-infrared transmission images of a Type-C connector. Perform high-precision spatial registration on the acquired visible light reflection and near-infrared transmission images, and stitch the registered visible light image data and near-infrared image data in the channel dimension to generate a four-channel digital image that simultaneously contains the external surface texture information and internal structural information of the connector as the image to be detected.

[0055] S20. Input the image to be detected into a pre-trained multi-self-supervised feature learning network. This network is pre-trained using an unlabeled image database containing various physical connectors such as USB-A, HDMI, and Type-C, and employs a contrastive learning task based on decoupling cross-spectral consistency and spectral specificity. The input image to be detected is processed through the representation perturbation learning branch in this network, and a first intermediate feature map is output, which encodes perturbation invariance features that are robust to changes in external illumination and noise interference. The same image to be detected is processed through the structure association learning branch in this network, and a second intermediate feature map is output, which encodes the spatial positional relationships and assembly constraints between different components such as pins and housings in the image.

[0056] S30. A dynamic gating fusion method based on channel information entropy and spatial structure saliency is adopted to fuse the first intermediate feature map and the second intermediate feature map; the information entropy value of the first intermediate feature map and the second intermediate feature map in each channel is calculated respectively; the two information entropy vectors are concatenated and then normalized by scaling and the Softmax function to generate the channel weight vector; the spatial gradient magnitude of the first intermediate feature map and the second intermediate feature map are extracted respectively; the two gradient magnitude matrices are multiplied element-wise and then input into the Sigmoid function to generate the spatial attention mask matrix; according to the formula... The calculations are performed to generate a deep feature map that integrates perturbation invariance features and structural correlation features.

[0057] S40. Input the fused depth feature map into a detection head network constructed based on deformable convolutional kernels and multi-scale feature pyramid structure; the detection head network processes the input depth feature map, performs defect category discrimination and defect location regression calculation, and outputs the final defect recognition result; the defect recognition result clearly indicates that the defect category is "surface scratch" and provides the rectangular positioning box information of the defect defined by pixel coordinates (120, 300, 150, 320) in the original image to be detected.

[0058] S50. When the production line needs to introduce a brand-new square multi-pin special connector for inspection, this new model is defined as the target new category connector. Collect about 15 images of this target new category connector, which have been manually and accurately labeled with defect categories and locations, to form a limited labeled sample set. Based on the few-shot learning strategy, use this limited labeled sample set to perform supervised targeted adjustment and fine-tuning of the key parameters in the aforementioned pre-trained multi-self-supervised feature learning network, so that the whole method can quickly adapt to and accurately complete the defect detection task of the new category connector.

[0059] Figure 2 A schematic diagram of an embodiment of a connector defect identification system 200 provided by the present invention is shown. Figure 2 As shown, the connector defect identification system 200 includes: The acquisition module 210 is used to acquire an image to be detected that includes information about the outer surface and internal structure of the connector; Input module 220 is used to input the image to be detected into a pre-trained multi-self-supervised feature learning network, which is pre-trained using an unlabeled image set containing multiple categories of connectors through a self-supervised learning task. Extraction module 230 is used to process the image to be detected through the representation perturbation learning branch in the multiple self-supervised feature learning network and output a first intermediate feature map. The first intermediate feature map represents perturbation invariance features that are robust to surface reflection, imaging noise and small morphological changes. The detection module 240 is used to process the image to be detected through the structural association learning branch in the multiple self-supervised feature learning network and output a second intermediate feature map, wherein the second intermediate feature map represents the structural topology and assembly relationship features between different component regions in the image to be detected. The fusion module 250 is used to fuse the first intermediate feature map and the second intermediate feature map to generate a fused depth feature map. The identification module 260 is used to perform defect classification and localization processing based on the fused depth feature map, and to obtain and output the defect identification result corresponding to the image to be detected. The defect identification result includes the defect category and the location information of the defect in the image.

[0060] It should be noted that the beneficial effects of the connector defect identification system 200 provided in the above embodiments are the same as those of the connector defect identification method described above, and will not be repeated here. Furthermore, the system provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to the actual situation to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, and will not be repeated here.

[0061] The connector defect identification system 200 of the present invention may be a computer program (including program code) running on a computer device. For example, the connector defect identification system 200 of the present invention is an application software that can be used to execute the corresponding steps in the connector defect identification method of the present invention.

[0062] In some embodiments, the connector defect identification system 200 of the present invention can be implemented in a combination of hardware and software. As an example, the connector defect identification system 200 of the present invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the connector defect identification method of the present invention. For example, the processor in the form of a hardware decoding processor can be one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0063] The modules described in the embodiments of this invention can be implemented in software or hardware. The names of the modules are not, in some cases, limiting the scope of the module itself.

[0064] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the connector defect identification methods described above. That is, an electronic device according to an embodiment of the present invention may include, but is not limited to: a processor and a memory; the memory is used to store the computer program; the processor is used to execute the connector defect identification method shown in any embodiment of the present invention by calling the computer program.

[0065] In one alternative embodiment, an electronic device is provided, such as Figure 3 As shown, Figure 3 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present invention.

[0066] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this invention. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0067] Bus 4002 may include a path for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3The bus 4002 is represented by only one thick line, but this does not mean that there is only one bus or one type of bus.

[0068] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0069] The memory 4003 stores application code (computer program) for executing the present invention, and its execution is controlled by the processor 4001. The processor 4001 executes the application code stored in the memory 4003 to implement the content shown in the foregoing method embodiments.

[0070] Among them, electronic devices can also be terminal devices. A terminal device can be any terminal device that can install applications and access web pages through applications, including at least one of smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, smart TVs, and smart in-vehicle devices.

[0071] It should be noted that, Figure 3 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0072] An embodiment of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-described connector defect identification methods.

[0073] Alternatively, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, a floppy disk, and an optical data storage device, etc.

[0074] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the aforementioned connector defect identification method.

[0075] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0076] It should be understood that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0077] The computer-readable storage medium provided in this invention can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0078] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the method shown in the above embodiments.

[0079] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this invention is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-disclosed concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this invention.

[0080] It should be noted that the terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and represent a limitation on a specific order or sequence. Where appropriate, the order of use for similar objects can be interchanged so that the embodiments of this application described herein can be implemented in an order other than that shown or described.

[0081] Those skilled in the art will recognize that this invention can be implemented as a system, method, or computer program product. Therefore, this invention can be specifically implemented in the following forms: it can be entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, this invention can also be implemented as a computer program product contained in one or more computer-readable media, which includes computer-readable program code.

[0082] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for identifying connector defects, characterized in that, include: Acquire an image of the connector to be inspected, containing information about the connector's external surface and internal structure; The image to be detected is input into a pre-trained multi-self-supervised feature learning network, which is pre-trained using an unlabeled image set containing multiple categories of connectors through a self-supervised learning task. The image to be detected is processed by the representation perturbation learning branch in the multi-self-supervised feature learning network to output a first intermediate feature map. The first intermediate feature map represents a perturbation invariance feature that is robust to surface reflection, imaging noise and small morphological changes. The image to be detected is processed by the structural association learning branch in the multi-self-supervised feature learning network to output a second intermediate feature map, which represents the structural topology and assembly relationship features between different component regions in the image to be detected. The first intermediate feature map and the second intermediate feature map are fused to generate a fused depth feature map; Based on the fused depth feature map, defect classification and localization processing is performed to obtain and output the defect identification result corresponding to the image to be detected. The defect identification result includes the defect category and the location information of the defect in the image.

2. The connector defect identification method according to claim 1, characterized in that, The step of acquiring the image to be detected, which includes information about the outer surface and internal structure of the connector, includes: The visible light reflection image and near-infrared transmission image of the connector are acquired simultaneously by multispectral imaging, and the visible light reflection image and the near-infrared transmission image are spatially registered and channel stitched to generate the image to be detected.

3. The connector defect identification method according to claim 2, characterized in that, The self-supervised learning task includes a comparative learning task based on the decoupling of cross-spectral consistency and spectral specificity.

4. The connector defect identification method according to claim 1, characterized in that, The step of fusing the first intermediate feature map and the second intermediate feature map includes: A dynamic gating fusion method based on channel information entropy and spatial structure saliency is adopted.

5. The connector defect identification method according to claim 1, characterized in that, The steps for performing defect classification and location processing include: A detection head network based on deformable convolution and multi-scale feature pyramids is used to process the fused depth feature map.

6. The connector defect identification method according to claim 1, characterized in that, The method further includes: After the multi-self-supervised feature learning network is pre-trained, the network parameters are adjusted in a targeted manner using limited labeled samples of the target new category connector based on a few-shot learning strategy.

7. The connector defect identification method according to claim 4, characterized in that, The dynamic gating fusion method based on channel information entropy and spatial structure saliency generates the fused deep feature map by jointly optimizing the channel weight vector and spatial attention mask. The formula for calculating the fused depth feature map is as follows: In the formula, This represents the first intermediate feature map. This represents the second intermediate feature map. This represents the fused depth feature map; This represents the channel weight vector, whose dimension is equal to the number of channels in the feature map. The calculation method is the same: This represents the calculation of the information entropy of the input feature map in each channel. This represents a vector concatenation operation. This is the scaling factor; The spatial attention mask matrix is ​​calculated as follows: This indicates an operation to extract the spatial gradient magnitude from the input feature map. This indicates element-wise multiplication; This represents the operation of converting a vector into a diagonal matrix.

8. A connector defect identification system, characterized in that, include: The acquisition module is used to acquire an image to be inspected that contains information about the external surface and internal structure of the connector; The input module is used to input the image to be detected into a pre-trained multi-self-supervised feature learning network, which is pre-trained using an unlabeled image set containing multiple category connectors through a self-supervised learning task. The extraction module is used to process the image to be detected through the representation perturbation learning branch in the multiple self-supervised feature learning network and output a first intermediate feature map. The first intermediate feature map represents perturbation invariance features that are robust to surface reflection, imaging noise and small morphological changes. The detection module is used to process the image to be detected through the structural association learning branch in the multiple self-supervised feature learning network and output a second intermediate feature map, which represents the structural topology and assembly relationship features between different component regions in the image to be detected. The fusion module is used to fuse the first intermediate feature map and the second intermediate feature map to generate a fused depth feature map; The identification module is used to perform defect classification and localization processing based on the fused depth feature map, and to obtain and output the defect identification result corresponding to the image to be detected. The defect identification result includes the defect category and the location information of the defect in the image.

9. An electronic device, characterized in that, The electronic device includes a processor coupled to a memory storing at least one computer program, which is loaded and executed by the processor to enable the electronic device to implement the connector defect identification method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which, when executed by a processor, implements the connector defect identification method as described in any one of claims 1 to 7.

Images