Intelligent fusion terminal defect detection method and detection system adaptive to workshop

Abstract

The invention relates to a workshop-adaptive intelligent fusion terminal defect detection method and a workshop-adaptive intelligent fusion terminal defect detection system in the technical field of electric power detection. The intelligent fusion terminal defect detection method comprises the following steps: collecting appearance image data, electrical parameter data and structure size data of a target, converting the data into corresponding channel characteristics, and inputting the channel characteristics into a defect identification neural network to obtain a result whether a defect exists or not and a defect type. Through the full-process design of multi-source data acquisition, lightweight intelligent detection and workshop equipment adaptation, the improved defect recognition neural network and the embedded attention module are adopted, so that the detection efficiency is remarkably improved, the workshop production line takt is completely matched, meanwhile, the detection precision and the coverage range are greatly expanded, the product reject ratio is reduced, and the product quality is improved. Moreover, the adaptability and robustness are enhanced through the lightweight neural network design, and the multi-scene production requirements are met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of power detection, in particular to a defect detection method and a detection system for an intelligent fusion terminal adapted to a workshop. BACKGROUND

[0002] The intelligent fusion terminal plays an important role in edge computing and is the core equipment of the low-voltage distribution Internet of Things. It can realize the organic integration of the power grid, communication network, Internet of Things and other systems in the distribution area, thereby improving the informatization level of the distribution area and providing strong support for the intelligent management and service of the distribution area. It can realize the full data collection of electricity, water, gas and heat energy consumption information, monitor the load operation status and power quality of the distribution area in real time and accurately, and quickly judge and locate faults based on the precise topology identification technology of the distribution area, greatly improving the efficiency of power grid operation and maintenance; it can also deeply support the unified management and coordinated control of diversified loads such as distributed photovoltaic, energy storage, and charging piles, and realize the intelligent autonomy of the distribution area by combining load forecasting and cloud-edge collaboration technology. However, in the current production process of intelligent fusion terminals, there are still many problems and limitations in the defect detection link. Based on the current production scenario of intelligent fusion terminals in the manufacturing industry, the current mainstream defect detection technology can be divided into three categories: (1) manual visual inspection + instrument-assisted inspection technology. This is the traditional technology adopted by most small and medium-sized power grid equipment manufacturers. (2) traditional machine vision inspection technology. Some large-scale manufacturers have introduced inspection technology based on "traditional algorithm + industrial camera". (3) Conventional deep learning detection technology. In recent years, some leading companies have tried to adopt mainstream deep learning models, such as YOLOv5 and Faster R-CNN. The above-mentioned existing technologies all have key problems that are difficult to adapt to the production scenarios of intelligent integrated terminals in the power grid manufacturing industry. The specific defects are as follows: (1) Manual visual inspection + instrument-assisted inspection technology: low efficiency, poor accuracy, and high cost. (2) Traditional machine vision inspection technology: weak generalization ability and can only detect appearance defects. Traditional machine vision relies on preset algorithms to extract fixed features (such as edges and gray levels). If the shape of terminal defects changes (such as different scratch angles and different degrees of poor soldering), or the lighting in the production environment changes (such as fluctuations in workshop lighting), the algorithm is prone to failure and cannot adapt to the detection needs of multiple scenarios. For example, Canny edge detection is prone to misjudging normal component outlines as scratches under strong light, with a misjudgment rate of over 25%. Furthermore, it cannot identify electrical performance defects (such as indicator light failures, weak communication module signals) or hidden structural defects (such as poor contact caused by internal soldering of components), requiring manual instrument detection, and cannot achieve "one-stop full defect detection". (3) Conventional deep learning detection technology: high equipment requirements, difficult deployment, and large resource consumption. Traditional YOLO series models and Faster R-CNN models require high-performance GPUs (such as RTX3070 and above) for inference, while most equipment in the workshop is an embedded terminal with low computing power, which cannot run such models; if forced to deploy, the inference speed is far lower than the real-time requirements of the production line. If a high-performance industrial server is used to deploy the model, the cost of a single server is high, and an additional data transmission link (connection between the server and the production line camera / sensor) needs to be built, increasing hardware investment. At the same time, the server requires professional maintenance (such as model updates and troubleshooting), further increasing enterprise costs.In addition, conventional deep learning detection techniques have low accuracy in identifying subtle defects (such as cold solder joints). Summary of the Invention

[0003] To address the technical problems of low detection accuracy, narrow detection range, and inability to adapt to low-computing-power equipment in workshops in existing intelligent fusion terminal defect detection technologies, this invention provides an intelligent fusion terminal defect detection method and detection system.

[0004] This invention is achieved through the following technical solution: In a first aspect, the present invention also proposes a defect detection method for an intelligent fusion terminal adapted to the workshop, which includes: The system collects the target's appearance image data, electrical parameter data, and structural dimension data, converts them into corresponding channel features, and inputs them into the defect recognition neural network to obtain the result of whether there is a defect and the defect category.

[0005] The defect recognition neural network comprises a backbone network and a detection head. The backbone network, MobileNetV2, includes: a shallow bottleneck layer, a middle bottleneck layer, and a deep bottleneck layer connected sequentially; several attention modules embedded in the middle and deep bottleneck layers; appearance image data, electrical parameter data, and structural dimension data are input into the shallow bottleneck layer to obtain shallow features representing basic defects; shallow features are input into the middle bottleneck layer to obtain middle-level features representing structural layout; middle-level features are input into the deep bottleneck layer to obtain deep features representing subtle defects; and attention modules are used to focus on processing deep features. The detection head includes a feature pyramid and an output layer; the feature pyramid is used for multi-task simultaneous extraction of shallow, middle, and deep features, and feature concatenation; the output layer processes the concatenated features and outputs the result of whether a defect exists and the defect category.

[0006] Secondly, this invention also proposes a workshop-adaptive intelligent fusion terminal defect detection system, which uses the workshop-adaptive intelligent fusion terminal defect detection method described in the first aspect. The intelligent fusion terminal defect detection system includes: a data acquisition module and a defect detection module. The data acquisition module is used to acquire the target's appearance image data, electrical parameter data, and structural dimension data, and convert them into corresponding channel features. The defect detection module is used to input the channel features into an improved defect recognition neural network to obtain a result indicating whether a defect exists and the defect category. The defect recognition neural network includes a backbone network and a detection head. The backbone network is MobileNetV2, which includes a shallow bottleneck layer, a middle bottleneck layer, and a deep bottleneck layer connected in sequence. Several attention modules are embedded in the middle and deep bottleneck layers. Appearance image data, electrical parameter data, and structural dimension data are input into the shallow bottleneck layer to obtain shallow features representing basic defects. The shallow features are input into the middle bottleneck layer to obtain middle features representing the structural layout. The middle features are input into the deep bottleneck layer to obtain deep features representing subtle defects. The attention modules are used to focus on processing the deep features. The detection head includes a feature pyramid and an output layer. The feature pyramid is used to extract shallow, middle, and deep features simultaneously across multiple tasks and perform feature concatenation. The output layer is used to process the concatenated features and output the result of whether there is a defect and the defect category.

[0007] Thirdly, the present invention also proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the intelligent fusion terminal defect detection method for the adaptive workshop in the first aspect.

[0008] The beneficial effects of this invention are as follows: 1. This invention adopts a full-process design of "multi-source data acquisition → lightweight intelligent detection → workshop equipment adaptation", and uses an improved defect recognition neural network and embedded attention module to significantly improve detection efficiency, fully match the pace of the workshop production line, and greatly expand detection accuracy and coverage, reduce product defect rate. In addition, the lightweight neural network design enhances adaptability and robustness, and meets the production needs of multiple scenarios.

[0009] 2. This invention constructs a "multi-source data fusion detection system" that is compatible with data such as appearance images, electrical parameters (current, voltage), and structural dimensions. It achieves full coverage detection of three types of defects: "appearance + electrical + hidden structure". It does not require additional manual labor or other detection equipment, simplifying the detection process. At the same time, by optimizing the convolutional structure, compressing the neural network volume, and quantizing the processing, the neural network is small in size, has low memory usage, and fast inference speed, making it suitable for low-computing-power embedded devices in the workshop without the need to purchase additional high-performance servers.

[0010] 3. This invention breaks through the "manual-dependent" inspection mode, realizing a fully closed-loop inspection process through automation technology, reducing the inspection time of a single terminal, ensuring that inspection efficiency is synchronized with the production line cycle time, and eliminating production bottlenecks. Furthermore, compared to traditional machine vision inspection, this invention optimizes the feature focusing mechanism and model structure to improve defect detection accuracy to ≥98%, reduce the missed detection rate of minor defects to ≤2%, and minimize the impact of environmental interference (such as light fluctuations and equipment vibration) on inspection accuracy, achieving "high-precision identification of all types of defects." Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart of a method for detecting defects in intelligent fusion terminals adapted to workshops; Figure 2 This is the architecture diagram of one of the bottleneck modules; Figure 3 This is the architecture diagram of the bottleneck module embedded in the attention module; Figure 4 This is a flowchart of the detection head's processing flow. Detailed Implementation

[0013] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0014] It should be noted that when a component is said to be "installed on" another component, it can be directly on the other component or it may be in a component that is centered on it. When a component is said to be "set on" another component, it can be directly set on the other component or it may also be in a component that is centered on it. When a component is said to be "fixed to" another component, it can be directly fixed to the other component or it may also be in a component that is centered on it.

[0015] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the specification of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "or / and" as used herein includes any and all combinations of one or more of the associated listed items.

[0016] This invention addresses the core pain points of defect detection in intelligent integrated terminals in the power grid manufacturing industry. It constructs a comprehensive technical solution encompassing "multi-source data acquisition → lightweight intelligent detection → workshop equipment adaptation." Through collaborative design of "hardware + algorithm + software," it achieves the detection goals of "high efficiency, accuracy, low cost, and easy deployment." Based on this design concept, this embodiment provides a workshop-adapted method for detecting defects in intelligent integrated terminals, comprising the following steps: acquiring the target's appearance image data, electrical parameter data, and structural dimension data, converting them into corresponding channel features, and inputting them into an improved defect recognition neural network to obtain the result of whether a defect exists and the defect category.

[0017] In this step, the acquired multimodal data solves the problem of "narrow detection range," thus covering multiple types of defects without the need for additional manual labor or other inspection equipment, simplifying the inspection process. Specifically, appearance image data can be acquired using industrial cameras. 5-megapixel industrial cameras are installed on both sides of the inspection station to capture images of the terminal's casing, interface areas, welding areas, etc., from the front and side, respectively. A ring light is used to automatically adjust brightness according to workshop lighting, eliminating the effects of lighting fluctuations and avoiding image blurring. This industrial camera can be connected to an industrial embedded terminal, allowing for the setting of appropriate frame rates and resolutions to ensure clear images and efficient acquisition. Electrical parameter data is acquired in real-time using industrial-grade current and voltage sensors to collect the terminal's operating current and output voltage during power-on testing, ensuring the capture of instantaneous electrical anomalies. Structural dimensional data is acquired using laser displacement sensors, which focus on key structural parameters such as terminal interface spacing and component mounting coordinates to identify defects such as component misalignment and omissions. The acquired multimodal data is also time-stamp aligned via hardware trigger signals, ensuring a one-to-one correspondence between "image-electrical-structural" data at the same detection moment. This provides a data foundation for multi-source fusion detection and avoids misjudgments of defects due to data asynchrony. Furthermore, to improve the quality of the acquired data, preprocessing is required: For appearance image data preprocessing, grayscale normalization and Gaussian filtering can be used to remove image noise, and adaptive histogram equalization can enhance the contrast of fine areas such as weld points and pins, solving the problems of overexposure in strong light and underexposure in weak light in traditional preprocessing. For electrical parameter data preprocessing, a sliding window can be used to smoothly filter high-frequency interference in current and voltage signals, and a threshold range for electrical parameters can be set to eliminate outliers exceeding the range, while retaining instantaneous fluctuation data (such as current peaks at startup) to avoid missing latent electrical defects. For structural dimension data preprocessing, coordinate calibration (with the terminal reference hole as the origin) can be performed on the coordinate data acquired by the laser displacement sensor to unify the measurement coordinate system of different terminal models. Outlier detection (3σ principle) can be used to identify structural data exceeding the standard size ±0.1mm and mark them as suspected defect points.

[0018] For scenarios where production workshop equipment has limited computing power and high real-time requirements, one of the core features of this invention is a lightweight, improved defect recognition neural network. The trained defect recognition neural network is imported into an embedded terminal. Its input is preprocessed multimodal data, and its output is the result of whether a defect exists and the defect category. It preferentially uses GPU inference, automatically switching to CPU when resources are insufficient. The architecture of this defect recognition neural network is described in detail below: Please refer to Figure 1The defect detection neural network includes a backbone network and a detection head. The backbone network is MobileNetV2, which consists of a shallow bottleneck layer, a middle bottleneck layer, and a deep bottleneck layer connected in sequence. Each of the shallow, middle, and deep bottleneck layers includes four bottleneck modules, numbered 1 through 12 in order of data processing. Attention modules are embedded in the bottleneck modules at layers 6, 9, and 12. All 12 bottleneck modules share the same architecture, including 1×1 convolutional units, 3×3 depthwise convolutional units, and 1×1 pointwise convolutional units. The 1×1 convolutional unit includes a 1×1 kernel, a Batch Normalization (BN) layer, and a ReLU6 activation layer. The 3×3 depthwise convolutional unit includes a 3×3 depthwise kernel, a BN layer, and a ReLU6 activation layer. The 1×1 pointwise convolutional unit includes a 1×1 kernel and a BN layer. The attention modules include channel attention units and spatial attention units. The channel attention unit includes: pooling layer po1, two convolutional layers conv1-conv2, two activation layers act1-act2, and a weighted layer wei1. The spatial attention unit includes: pooling layer po2, convolutional layer conv3, activation layer act3, and a weighted layer wei2. The detection head includes: a feature pyramid and an output layer. The output layer includes: convolutional layer conv4 and activation layer act4.

[0019] During data inference, this defect recognition neural network first collects and preprocesses appearance image data, electrical parameter data, and structural dimension data, then fuses them and inputs them into a shallow bottleneck layer to obtain shallow features representing basic defects. Next, these shallow features are input into a middle bottleneck layer to obtain middle-layer features representing the structural layout. Finally, these middle-layer features are input into a deep bottleneck layer to obtain deep features representing subtle defects. Further, such as... Figure 2 As shown, in each bottleneck module, a 1×1 convolutional unit increases the dimensionality of the input using a 1×1 convolutional kernel. A 3×3 depthwise convolutional unit performs depthwise convolution on the output of the 1×1 convolutional unit using a 3×3 depthwise convolutional kernel to extract features. A 1×1 pointwise convolutional unit then performs pointwise convolution on the output of the 3×3 depthwise convolutional unit to reduce dimensionality, and the output of the 1×1 pointwise convolutional unit is concatenated with the input residual of the 1×1 convolutional unit. The fourth bottleneck module outputs shallow features. The eighth bottleneck module outputs mid-level features. The twelfth bottleneck module outputs deep features. Figure 3As shown, the attention module focuses on processing deep features during this process. Its channel attention unit receives the output of the bottleneck module, calculates the importance weights of each channel feature, and strengthens features related to defects. The spatial attention unit receives the output of the channel attention unit and focuses on processing the spatial region where the defect is located. Further, in the channel attention unit, po1 is used to perform global average pooling on the input. conv1 is used to reduce the dimensionality of the pooling layer output. acr1 is used to introduce nonlinearity into the output of conv1. conv2 is used to increase the dimensionality of the output of acr1 to restore the number of channels. act2 is used to activate the output of conv2 to generate channel weights. wei1 is used to perform weighted calculation of each channel feature using the channel weights. In the spatial attention unit, po2 is used to perform channel-dimensional average pooling on the output of wei1. conv3 is used to extract spatial correlation features from the output of po2. act3 is used to activate the output of conv3 to generate spatial weights. wei2 is used to focus on processing the spatial region where the defect is located and output deep features using the spatial weights. The feature pyramid in the detection head simultaneously extracts shallow, medium, and deep features through multiple tasks and performs feature concatenation. In the output layer, conv4 performs feature extraction and dimensionality reduction on the concatenated features. act4 activates the output of conv4 and outputs the result of whether there is a defect and the defect category.

[0020] Based on the above description of the architecture and inference process of the defect recognition neural network, the improvements of this defect recognition neural network can be summarized as follows: optimization of depthwise separable convolution, adjustment of bottleneck structure, simplification of network layer number, embedding of attention mechanism, optimization of embedding position, and design of detection head.

[0021] (1) Depth-separable convolution optimization.

[0022] First, MobileNetV2 was chosen as the backbone network. Its core advantage lies in replacing traditional convolution with depthwise separable convolution, significantly reducing the number of parameters and computational cost, making it suitable for low-computing embedded devices in workshops. At the same time, the traditional 3×3 convolution is split into "3×3 depthwise convolution (convolution only on a single channel to extract local features of a single channel) + 1×1 point convolution (fusion of multi-channel features to achieve information interaction between channels)", reducing the number of parameters and computational cost accordingly, making it directly adaptable to industrial CPUs / GPUs in workshops.

[0023] (2) Bottleneck structure adjustment.

[0024] By leveraging the inverted residual bottleneck structure of MobileNetV2, the dimensionality of intermediate layer feature maps is further reduced while ensuring that the ability to extract defect features (such as grayscale changes in cold solder joints and interface edge contours) is not lost. This reduces the memory footprint of intermediate feature maps for a single input image and alleviates the pressure on device memory.

[0025] (3) The number of network layers is reduced.

[0026] For intelligent fusion terminal defect detection scenarios (defect types mainly include cosmetic scratches, interface misalignment, and component cold solder joints, with feature complexity lower than general target detection scenarios), the traditional MobileNetV2 bottleneck structure of 19 layers is simplified to 12 layers, removing the last 3 computationally intensive bottleneck modules. Simultaneously, the layered functions are clearly defined: shallow layers (layers 1-4) focus on extracting basic defect features such as edges and textures (e.g., scratch edges, component outlines). Middle layers (layers 5-8) extract structural features such as interface spacing and component layout. Deep layers (layers 9-12) extract subtle defect features such as cold solder joints and missing solder joints, ensuring that detection accuracy is not compromised while improving model inference speed by 40%.

[0027] (4) Attention mechanism embedding.

[0028] To address the issue of insufficient ability of lightweight models to identify subtle defects (such as component solder joints smaller than 0.1mm and bent pins), a lightweight attention module (CBAM, Convolutional Block Attention Module) is embedded after the mid-layer and deep bottleneck structures of MobileNetV2. This module focuses on defect features through a dual-dimensional approach of "channel + space". Specifically, the channel attention unit adopts a structure of "global average pooling (compressing spatial dimension, preserving channel feature importance) + 1×1 convolution dimensionality reduction + ReLU activation (introducing nonlinearity) + 1×1 convolution dimensionality increase (restoring the number of channels) + Sigmoid activation (generating channel weights)" to replace the fully connected layer in the traditional CBAM, reducing the number of parameters by 90%. By calculating the importance weights of each channel feature, it strengthens channel features related to defects (such as grayscale variation channels in solder joint areas and edge gradient channels in interface misalignment) and suppresses irrelevant background channels (such as solid color channels in defect-free areas of the terminal casing). The spatial attention unit employs a structure of "channel-dimensional average pooling + 3×3 convolution (extracting spatial correlation features, reducing the number of convolution kernels from 64 to 32, reducing computation by 50%) + Sigmoid activation (generating spatial weights)". By focusing on the spatial region where defects are located (such as the location of scratches on the terminal shell or the central region of component solder joints), the model's sensitivity to local minor defects is improved, avoiding interference from background information.

[0029] (5) Optimization of embedding location.

[0030] This defect recognition neural network embeds attention modules only after the bottleneck structures of MobileNetV2: layer 6 (at the end of the middle layer, strengthening structural feature focusing), layer 9 (in the middle of the deep layer, strengthening the initial recognition of minor defects), and layer 12 (at the end of the deep layer, strengthening the final output of minor defect features). This avoids the surge in computational cost caused by full network embedding (which would increase computational cost by more than 30%), while ensuring that key defect features are effectively enhanced. This embedding method can improve the recognition accuracy of minor defects (0.1-0.5mm).

[0031] (6) Detection head design.

[0032] To address the dual task requirements of intelligent fusion terminals—"defect presence determination" and "defect type classification"—a specially designed lightweight inspection head reduces computational resource consumption while meeting task requirements. Figure 4 As shown, for the optimization of the feature pyramid, shallow features are extracted through downsampling (output of the bottleneck module at layer 4; through max pooling downsampling, the shallow feature size is reduced from 224×224×C to 112×112×C, where C represents the number of channels). Mid-layer features are directly extracted (output of layer 8, 112×112×C). Deep features are extracted through upsampling (output of layer 12; through bilinear interpolation upsampling, the deep feature size is increased from 56×56×C to 112×112×C). After feature concatenation, the feature pyramid retains only two feature scales (112×112 and 56×56). Compared to traditional feature pyramids with 3-4 scales, the computational cost is reduced by 40%, while ensuring coverage of defects of different sizes: the 112×112 scale adapts to minor defects such as scratches less than 5mm and cold solder joints less than 1mm; the 56×56 scale adapts to larger defects such as scratches greater than 5mm and interface misalignments. To optimize the output layer, a lightweight output layer using "1×1 convolution + Softmax activation" is adopted to replace the fully connected layer of the traditional detection model. The output dimension is only "number of defect categories + 1 (presence or absence of a defect)". Taking five typical defects of a smart fusion terminal (appearance scratches, interface misalignment, component cold solder joints, missing solder joints, and indicator light malfunctions) as an example, the output dimension is 6 (5 defect categories + 1 category for presence or absence of a defect). This design reduces the number of parameters in the output layer to 1 / 10 of that in a fully connected layer, improving inference speed while avoiding the overfitting problem of fully connected layers.

[0033] In summary, this invention simplifies the backbone network structure and clearly defines the hierarchical feature extraction function (shallow layer extracts basic defect features, mid-layer extracts structural defect features, and deep layer extracts subtle defect features). Compared with existing traditional CNNs or unoptimized MobileNet, traditional methods suffer from high memory consumption and poor adaptability to low-computing-power devices. This design significantly reduces model memory consumption without sacrificing feature extraction capabilities, making it directly adaptable to embedded devices in workshops. Furthermore, by embedding simplified channel + spatial attention modules only at key locations in the mid- and deep layers of the network, without full network embedding, it avoids the problems of existing technologies either leading to a surge in computation due to full network embedding of attention modules or weak subtle defect recognition due to the absence of attention modules. This design strengthens feature focusing on subtle defects such as cold solder joints and bent pins while avoiding excessive computational increases, balancing detection accuracy and inference efficiency. The use of a low-scale feature pyramid (retaining only two core feature scales) coupled with a simplified output layer (replacing the traditional fully connected layer with a 1×1 convolution) avoids the computational redundancy of existing multi-scale fusion designs and the overfitting problem of fully connected output layers. This design reduces the computational cost and number of parameters for multi-scale fusion while avoiding overfitting, and can simultaneously meet the dual task requirements of "defect presence / absence + type classification".

[0034] On the other hand, this defect recognition neural network needs to be used after training. Addressing the issues of limited sample size and limited scenarios in workshop production data (fixed lighting and fixed workstations only), it employs a triple optimization approach—data augmentation, loss function design, and quantization compression—to improve the model's generalization ability and equipment adaptability. Specifically, the training methods for the defect recognition neural network include: First, a data augmentation method was employed, combining random flipping (horizontal / vertical flipping to expand angle diversity), brightness adjustment (simulating workshop lighting fluctuations), and electrical parameter perturbation, to expand the training sample size by three times. This effectively avoided model overfitting and improved adaptability to different workshop operating conditions. Next, the augmented training data was used to train the defect recognition neural network using a hybrid loss function of "cross-entropy loss + L1 loss." Cross-entropy loss ensured the accuracy of defect category determination, while L1 loss reduced defect location errors (such as scratch bounding boxes and coordinates of cold weld areas), ultimately achieving a location accuracy that met the workshop's requirements for precise defect location tracing. After training, quantization technology was used to compress the weight parameters in the defect recognition neural network into INT8 type, reducing the weight parameters from 32-bit floating-point numbers to 8-bit integers, significantly reducing the size of the defect recognition neural network. Simultaneously, quantization-aware training was used during training to simulate quantization errors and compensate for accuracy loss. The quantized defect recognition neural network achieved a detection accuracy of 5 typical defect types that met production quality control requirements and was well-suited to the limited storage and memory resources of workshop equipment. When testing new models of intelligent fusion terminals, the parameters of the original defect recognition neural network are loaded, and the shallow, medium, and deep bottleneck layers are frozen. The attention module and detection head are trained using only sample data from the new intelligent fusion terminal. Existing technologies require full network retraining or extensive sample fine-tuning, resulting in long adaptation cycles and high costs. This design can shorten the adaptation time for multiple terminal models, reduce the amount of sample collection, and lower workshop operation and maintenance costs.

[0035] In one instance, to address the issues of "difficult deployment and high cost," further optimizations were made in both hardware compatibility and software adaptation to ensure the stable implementation of this lightweight defect recognition neural network.

[0036] 1. Hardware compatibility design.

[0037] The data acquisition hardware uses a dual interface of "USB 3.0 + EtherNet", which can be directly connected to existing industrial cameras and sensors in the workshop without replacing hardware, reducing transformation costs. The detection algorithm supports "CPU / GPU adaptive scheduling", which automatically switches to CPU inference mode on industrial CPU equipment without GPU. The inference speed meets the production line cycle time requirements, avoiding the additional cost of forcibly purchasing GPU equipment.

[0038] 2. Software compatibility optimization.

[0039] Lightweight inspection software based on a defect recognition neural network has been developed, with an operating system adapted for embedded industrial terminals in workshops. The software includes a built-in "equipment status monitoring module" that displays CPU / GPU usage, memory consumption, and network transmission rate in real time.

[0040] Once the hardware and software are deployed, the entire testing process can be fully automated. The terminal under test is sent to the inspection station, triggering a detection signal, and the embedded terminal simultaneously starts data acquisition. The pre-processed data is input into the defect recognition neural network in the embedded terminal, which outputs a result indicating whether a defect exists and the defect category. After the result is output, an inspection report is automatically generated, containing key defect information. Based on the report, the production line sends the defective terminal to the rework station for re-inspection until it passes inspection.

[0041] In another embodiment, a smart fusion terminal defect detection system adapted to a workshop is proposed, which uses the smart fusion terminal defect detection method adapted to a workshop described in the above embodiment. The smart fusion terminal defect detection system includes: a data acquisition module and a defect detection module. The data acquisition module is used to acquire the target's appearance image data, electrical parameter data, and structural dimension data, and convert them into corresponding channel features. The defect detection module is used to input the channel features into an improved defect recognition neural network to obtain the result of whether there is a defect and the defect category. The defect recognition neural network includes: a backbone network and a detection head. The backbone network is MobileNetV2, which includes: a shallow bottleneck layer, a middle bottleneck layer, and a deep bottleneck layer connected in sequence. Several attention modules are embedded in the middle bottleneck layer and the deep bottleneck layer. Appearance image data, electrical parameter data, and structural dimension data are input into the shallow bottleneck layer to obtain shallow features representing basic defects. Shallow features are input into the middle bottleneck layer to obtain middle features representing the structural layout. Middle features are input into the deep bottleneck layer to obtain deep features representing minor defects. The attention modules are used to focus on processing the deep features. The detection head consists of a feature pyramid and an output layer. The feature pyramid is used for multi-task simultaneous extraction of shallow, mid-level, and deep features, and then performs feature concatenation. The output layer processes the concatenated features and outputs the result indicating whether there are defects and the defect category.

[0042] In another embodiment, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of the intelligent fusion terminal defect detection method for the adaptive workshop described in the above embodiments.

[0043] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0044] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method for defect detection using an intelligent fusion terminal adapted to a workshop, characterized in that, It includes: The system collects the target's appearance image data, electrical parameter data, and structural dimension data, converts them into corresponding channel features, and inputs them into the defect recognition neural network to obtain the result of whether there is a defect and the defect category. The defect recognition neural network includes: a backbone network and a detection head; The backbone network is MobileNetV2, which includes: a shallow bottleneck layer, a middle bottleneck layer, and a deep bottleneck layer connected in sequence; several attention modules embedded in the middle bottleneck layer and the deep bottleneck layer; appearance image data, electrical parameter data, and structural dimension data are input into the shallow bottleneck layer to obtain shallow features representing basic defects; shallow features are input into the middle bottleneck layer to obtain middle features representing structural layout; middle features are input into the deep bottleneck layer to obtain deep features representing subtle defects; attention modules are used to focus on processing deep features. The detection head includes a feature pyramid and an output layer. The feature pyramid is used to extract shallow, medium, and deep features simultaneously across multiple tasks and to perform feature concatenation. The output layer is used to process the concatenated features and output the result of whether there are defects and the defect category.

2. The method for detecting defects in intelligent fusion terminals adapted to workshops according to claim 1, characterized in that, The appearance image data is preprocessed by grayscale normalization and Gaussian filtering; the electrical parameter data is preprocessed by sliding window smoothing filtering; and the structural dimension data is preprocessed by coordinate calibration and outlier detection.

3. The method for detecting defects in intelligent fusion terminals adapted to workshops according to claim 1, characterized in that, The shallow bottleneck layer, the middle bottleneck layer, and the deep bottleneck layer all include several bottleneck modules connected in series. The bottleneck module includes: 1×1 convolutional units, 3×3 depthwise convolutional units, and 1×1 pointwise convolutional units; A 1×1 convolutional unit is used to increase the dimensionality of the input; A 3×3 depthwise convolutional unit is used to perform depthwise convolution on the output of a 1×1 convolutional unit to extract features; A 1×1 point convolutional unit is used to perform point-by-point convolution on the output of a 3×3 depth convolutional unit to reduce dimensionality, and the output of the 1×1 point convolutional unit is connected to the input residual of the 1×1 convolutional unit.

4. The method for detecting defects in intelligent fusion terminals adapted to workshops according to claim 3, characterized in that, The shallow bottleneck layer, the middle bottleneck layer, and the deep bottleneck layer each contain 4 bottleneck modules; in the order of data processing, they are the bottleneck modules of layers 1 to 12; the fourth bottleneck module outputs shallow features. The bottleneck module at layer 8 outputs mid-layer features; The bottleneck module at layer 12 outputs deep features; The attention module is embedded in the bottleneck modules of layers 6, 9, and 12.

5. The method for detecting defects in intelligent fusion terminals adapted to workshops according to claim 4, characterized in that, The attention module includes: channel attention unit and spatial attention unit; The channel attention unit receives the output of the bottleneck module, calculates the importance weights of each channel feature, and strengthens the features related to the defect. The spatial attention unit is used to receive the output of the channel attention unit and focus on processing the spatial region where the defect is located.

6. The method for detecting defects in intelligent fusion terminals adapted to workshops according to claim 5, characterized in that, The channel attention unit includes: a pooling layer po1, two convolutional layers conv1 to conv2, two activation layers act1 to act2, and a weighted layer wei1; Wherein, po1 is used to perform global average pooling on the input; conv1 is used to reduce the dimensionality of the output of the pooling layer; acr1 is used to introduce nonlinearity into the output of conv1; conv2 is used to increase the dimensionality of the output of acr1 to restore the number of channels; act2 is used to activate the output of conv2 to generate channel weights; wei1 is used to perform weighted calculation on the features of each channel through the channel weights; The spatial attention unit includes: pooling layer po2, convolutional layer conv3, activation layer act3, and weighted layer wei2; Among them, po2 is used to perform channel-dimensional average pooling on the output of wei1; conv3 is used to extract spatial correlation features from the output of po2; act3 is used to activate the output of conv3 to generate spatial weights; and wei2 is used to focus on the spatial region where the defect is located and output deep features through spatial weights.

7. The method for defect detection of intelligent fusion terminals adapted to workshops according to claim 1, characterized in that, In the feature pyramid, shallow features are extracted through downsampling; mid-level features are extracted directly. Deep features are extracted through upsampling; and shallow, mid-level, and deep features are unified to the same feature scale. The feature pyramid outputs two feature scales; The output layer includes: a convolutional layer (conv4) and an activation layer (act4); conv4 is used to extract features and reduce dimensionality from the output of the feature pyramid; `act4` is used to activate the output of `conv4` to obtain the result of whether there is a defect and the defect category.

8. The method for detecting defects in intelligent fusion terminals adapted to workshops according to claim 1, characterized in that, The defect identification neural network is used after training, and its training methods include: The sample data were enhanced using random flipping, brightness adjustment, and current perturbation. The defect recognition neural network is trained using enhanced training data. After training, quantization technology is used to compress the parameters in the defect recognition neural network into INT8 type. During the training process, quantization perception training is used to simulate quantization error in order to compensate for the loss of accuracy. When testing the new model of intelligent fusion terminal, the parameters of the original defect recognition neural network are loaded, the shallow bottleneck layer, the middle bottleneck layer, and the deep bottleneck layer are frozen, and the attention module and detection head are trained only using the sample data of the new model of intelligent fusion terminal.

9. A smart fusion terminal defect detection system adapted to workshops, characterized in that, It uses the intelligent fusion terminal defect detection method for adapting workshops as described in any one of claims 1 to 8; The intelligent fusion terminal defect detection system includes: The acquisition module is used to acquire the target's appearance image data, electrical parameter data, and structural dimension data, and convert them into corresponding channel features; The defect detection module is used to input channel features into an improved defect recognition neural network to obtain the result of whether a defect exists and the defect category. The defect recognition neural network includes a backbone network and a detection head. The backbone network is MobileNetV2, which includes a shallow bottleneck layer, a middle bottleneck layer, and a deep bottleneck layer connected in sequence. Several attention modules are embedded in the middle and deep bottleneck layers. Appearance image data, electrical parameter data, and structural dimension data are input into the shallow bottleneck layer to obtain shallow features representing basic defects. Shallow features are input into the middle bottleneck layer to obtain middle features representing structural layout. Middle features are input into the deep bottleneck layer to obtain deep features representing subtle defects. The attention modules are used to focus on processing the deep features. The detection head includes a feature pyramid and an output layer. The feature pyramid is used for multi-task simultaneous extraction of shallow, middle, and deep features and feature concatenation. The output layer processes the concatenated features and outputs the result of whether a defect exists and the defect category.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the intelligent fusion terminal defect detection method for the adaptive workshop as described in any one of claims 1 to 8.

Images