Light-weight industrial device surface defect detection method, system, equipment and medium

Abstract

The invention discloses a lightweight industrial device surface defect detection method, system and equipment and a medium, and the method comprises the steps: obtaining any one of or a combination of two of an image and point cloud data of a normal sample in an industrial scene, and training a pre-established teacher model; constructing a student model, generating knowledge distillation through masking, migrating teacher features to a student network, and training the student model; performing channel sorting pruning on the trained student model to obtain a pruning model; and inputting a to-be-detected sample collected in real time into the pruning model, and judging whether the to-be-detected sample has defects or not. According to the method, the knowledge distillation technology and the model pruning technology are combined, through iterative optimization, the complexity of the model is greatly reduced while high precision is kept, and the limitation of a single lightweight method on a multi-modal defect detection task is effectively solved. According to the invention, the target of deploying a high-performance multi-modal defect detection model on edge equipment can be realized, and meanwhile, relatively high detection precision and modal loss robustness are kept.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and industrial quality inspection, and in particular to a method, system, equipment and medium for detecting surface defects of lightweight industrial devices. Background Art

[0002] Driven by smart manufacturing and Industry 4.0, vision-based defect detection has become a core link in ensuring product quality and production safety. If surface defects in key components such as aircraft engine blades, 3C structural parts, and lithium electrode sheets are not discovered in a timely manner, they may cause cascading failures or even catastrophic accidents. In recent years, large multimodal models represented by the Vision Transformer (ViT) and the Point Transformer have significantly improved defect detection accuracy: ViT captures long-range texture dependencies through a global self-attention mechanism, while the Point Transformer mines local geometric relationships in 3D point clouds. The fusion of the two can fully utilize the complementary information of color, texture, and depth, achieving an AUROC (Area Under the Receiver Operating Characteristic Curve) of over 90% on benchmark datasets such as MVTec-3D AD.

[0003] However, the above high-precision models often come with huge computational and storage overhead:

[0004] (1) Excessive number of parameters: The original multimodal teacher model has more than 180M parameters, an inference time of approximately 12ms / sample, and a video memory usage of nearly 5GB;

[0005] (2) Limited edge deployment: If deployed directly at the edge of the production line (such as the NVIDIA Jetson series, Raspberry Pi, or PLC-level industrial computers), it will face real-time bottlenecks caused by insufficient resources (4–16GB shared memory, 10–30TOPS computing power);

[0006] (3) Failure of lightweight methods: Traditional pruning, quantization, and knowledge distillation are mainly designed for ImageNet classification or single-modal detection tasks, and are difficult to directly migrate to "image + point cloud" multimodal defect detection; rough pruning easily leads to the loss of geometric details, and the distillation strategy ignores cross-modal alignment, ultimately causing a cliff-like drop in accuracy (AUROC drop > 5%);

[0007] (4) Insufficient robustness to modality loss: If image or point cloud data is missing (such as sensor failure or occlusion) during the inference phase, the performance of existing methods will degrade significantly (AUROC decreases by >10%).

[0008] In summary, there is an urgent need for a "high-precision-lightweight" collaborative method for multimodal defect detection, which can significantly reduce the number of parameters and inference delay while maintaining high sensitivity to tiny defect areas, and has modal missing robustness and real-time edge deployment capabilities. Summary of the Invention

[0009] The purpose of the present invention is to solve the problems faced by existing industrial device surface defect detection models in edge deployment, such as slow reasoning speed, large number of parameters, and high computing resource usage. In particular, under actual working conditions where multimodal input is incomplete (such as missing images or point clouds), traditional large models are difficult to achieve real-time detection on resource-constrained devices. To this end, a lightweight industrial device surface defect detection method, system, equipment and medium are provided, which combine knowledge distillation and model pruning technology to achieve model structure compression and improved reasoning efficiency without significantly reducing detection accuracy. The method is suitable for edge scenarios such as industrial quality inspection and production line monitoring.

[0010] In order to achieve the above object, the present invention has the following technical solutions:

[0011] In a first aspect, a method for detecting surface defects of lightweight industrial devices is provided, comprising:

[0012] Obtain either or a combination of image and point cloud data of normal samples in industrial scenes to train the pre-established teacher model;

[0013] Build a student model, transfer teacher features to the student network through masked generative knowledge distillation, and train the student model;

[0014] Perform channel sorting and pruning on the trained student model to obtain a pruned model;

[0015] The samples to be tested collected in real time are input into the pruning model to determine whether the samples to be tested have defects.

[0016] As a preferred solution, the teacher model is established using a Transformer-based multimodal defect detection network, which is composed of a cascade of an image encoder, a point cloud encoder, and a cross-modal attention fusion module;

[0017] The image encoder adopts the Vision Transformer structure to divide the input image into image blocks of fixed size and capture global texture features through a multi-layer self-attention mechanism;

[0018] The point cloud encoder adopts the deep point cloud hierarchical feature learning network PointNet++ hierarchical structure to extract the geometric features of the point cloud through farthest point sampling and local geometric aggregation;

[0019] The cross-modal attention fusion module deeply fuses image features with point cloud features through a cross-attention mechanism, and outputs a unified multimodal feature representation for subsequent defect detection.

[0020] As a preferred solution, the student model is established by using a network structure that is a lightweight version of the teacher model, and the number of network layers and channels are both smaller than those of the teacher model;

[0021] During the training phase, the intermediate feature map input to the student model is randomly masked with pixels according to a preset masking ratio to obtain masked features;

[0022] Map the masked features to reconstructed features consistent with the feature dimensions of the corresponding layer of the teacher model;

[0023] Calculate the distillation loss Loss between the reconstructed features and the teacher features diss ;

[0024] Joint original defect detection loss Loss ori Distillation loss diss Train the student model.

[0025] As a preferred solution, in the step of performing random pixel masking on the intermediate feature map input to the student model according to a preset masking ratio, the masking ratio is a fixed value or a dynamic value between 10% and 80%.

[0026] As a preferred solution, the masked features are mapped to reconstructed features with the same feature dimension as the corresponding layer of the teacher model through a masked generation and reconstruction module, and the masked generation and reconstruction module consists of a sequentially connected 1×1 convolution layer, a 3×3 convolution layer and a ReLU activation function.

[0027] As a preferred solution, the distillation loss between the reconstructed features and the teacher features is Loss diss L2 distance or cosine similarity is used for measurement.

[0028] As a preferred solution, the step of performing channel sorting and pruning on the trained student model to obtain a pruned model includes:

[0029] Divide the student model's network into several segments based on producer-consumer relationships;

[0030] Construct an undirected graph for each network segment, where nodes represent channel sets and edges represent channel sharing relationships;

[0031] The maximum reward path algorithm is used to determine the channel retention order. The maximum reward path algorithm is implemented by solving the maximum reward value of the acyclic path through dynamic programming, and the convolution kernels and feature maps are rearranged in the corresponding order.

[0032] Delete redundant channels at a preset pruning rate to obtain a pruned model;

[0033] In the step of inputting the sample to be detected collected in real time into the pruning model and determining whether the sample to be detected has defects, a reconstruction error or a density estimation anomaly score of the sample to be detected is calculated, where the anomaly score is obtained by calculating the mean square error between the input image and the reconstructed image or a probability density based on a normalized flow;

[0034] When the anomaly score is greater than the threshold, it is determined that the sample to be tested has defects.

[0035] In a second aspect, a lightweight industrial device surface defect detection system is provided, comprising:

[0036] The teacher model training module is used to obtain either images or point cloud data of normal samples in industrial scenarios, or a combination of the two, to train the pre-established teacher model;

[0037] The student model training module is used to build a student model, transfer teacher features to the student network through masked generative knowledge distillation, and train the student model;

[0038] The pruned model acquisition module is used to perform channel sorting pruning on the trained student model to obtain a pruned model;

[0039] The defect judgment module is used to input the samples to be detected collected in real time into the pruning model to determine whether the samples to be detected have defects.

[0040] According to a third aspect, an electronic device is provided, including:

[0041] A memory storing at least one instruction; and a processor executing the instruction stored in the memory to implement the method for detecting surface defects of lightweight industrial devices.

[0042] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores at least one instruction, and the at least one instruction is executed by a processor in an electronic device to implement the surface defect detection method of lightweight industrial devices.

[0043] Compared with the prior art, the present invention has at least the following beneficial effects:

[0044] Existing high-precision multimodal defect detection models have complex structures and large number of parameters, making them difficult to deploy directly on resource-constrained edge devices. Simple model compression methods cannot effectively handle the characteristics of multimodal data, resulting in a significant decrease in accuracy and insufficient robustness. The present invention proposes a lightweight industrial device surface defect detection method that combines knowledge distillation and model pruning. Through masked generative knowledge distillation, teacher features are transferred to the student network, which not only reduces the number of parameters in the student model but also improves the model's robustness to incomplete inputs, addressing the limitations of traditional knowledge distillation in multimodal tasks. The trained student model is pruned by channel sorting to obtain a pruned model. The samples to be detected collected in real time are input into the pruned model to determine whether the samples to be detected have defects. This can significantly reduce the number of parameters and computational complexity while maintaining model performance. The present invention combines knowledge distillation and model pruning techniques. Through iterative optimization, the complexity of the model is significantly reduced while maintaining high accuracy. This joint optimization strategy effectively addresses the limitations of single lightweight methods in multimodal defect detection tasks. The present invention can achieve the goal of deploying a high-performance multimodal defect detection model on edge devices while maintaining high detection accuracy and modality loss robustness. Through practical verification, the model size can be compressed to 1 / 2 to 1 / 3 of the original model, and the inference speed is increased by more than 30%; on the three-dimensional anomaly detection dataset MVTec-3DAD, the image-area under the curve (receiver operating characteristic curve) Image-AUROC only decreased by 1% to 2%, which is still better than the traditional lightweight model; in the case of missing images or point clouds, it can still maintain robustness; and it is edge-friendly and supports real-time operation on resource-constrained devices such as the embedded AI computing platform Jetson Nano and Raspberry Pi; at the same time, the surface defect detection method of lightweight industrial devices of the present invention has the ability to continue to expand, and through the teacher model-student model closed loop, it supports the rapid adaptation of new tasks of the production line.

[0045] Furthermore, the masked generation and reconstruction module in this embodiment of the present invention consists of a sequentially connected 1×1 convolutional layer, a 3×3 convolutional layer, and a ReLU activation function. The 1×1 convolutional layer adjusts the number of channels and reduces the number of parameters; the 3×3 convolutional layer captures local spatial information, helping to recover masked features; and the activation function introduces nonlinearity, enhancing the model's expressiveness. This simple structural design minimizes computational overhead while ensuring effective reconstruction.

[0046] Furthermore, the distillation loss between the reconstruction feature and the teacher feature in the embodiment of the present invention is Loss dissThe L2 distance or cosine similarity is used for measurement, the L2 distance can directly measure the Euclidean distance between the feature vectors, and is suitable for capturing absolute difference; and the cosine similarity can measure the similarity of the direction of the feature vector, and is not sensitive to scale, and is suitable for capturing relative difference. By selecting a suitable measurement method, the student model can be better guided to learn the feature representation of the teacher model.

[0047] Further, the maximum reward path algorithm is used to determine the channel reservation order, the maximum reward path algorithm is realized by dynamic programming to solve the maximum reward value of the loop-free path, and the convolution kernel and the feature map are rearranged in the corresponding order, the dynamic programming algorithm can efficiently solve the optimal channel reservation order, and the calculation overhead of the brute force search is avoided. By solving the maximum reward value of the loop-free path, the important channels can be reserved while minimizing the memory copy overhead. This method can realize more fine and efficient pruning under the premise of ensuring the model performance. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced below, and it should be understood that the following drawings only show some embodiments of the present application, and other related drawings can also be obtained by those skilled in the art without creative labor.

[0049] Figure 1 The overall architecture diagram of the knowledge distillation generated based on masking of the embodiments of the present application is shown in the figure.

[0050] Figure 2 The student model network structure diagram constructed by the embodiments of the present application is shown in the figure.

[0051] Figure 3 The channel ordering pruning method flowchart of the embodiments of the present application is shown in the figure.

[0052] Figure 4 The ordering pruning method of the shared channel of the embodiments of the present application is shown in the figure.

[0053] Figure 5 The multi-modal defect detection feature extraction flowchart of the embodiments of the present application is shown in the figure.

[0054] Figure 6 The multi-modal defect detection feature fusion and decision layer fusion flowchart of the embodiments of the present application is shown in the figure.

[0055] Figure 7 The defect positioning effect example diagram of the embodiments of the present application for MVTec-3D AD 10-class objects is shown in the figure.

[0056] Figure 8This is a comparison curve of Image-AUROC in the point cloud missing scenario before and after lightweighting according to an embodiment of the present invention;

[0057] Figure 9 This is a comparison curve of Image-AUROC in the image missing scenario before and after lightweighting according to an embodiment of the present invention;

[0058] Figure 10 This is a comparison graph of AUPRO in the point cloud missing scenario before and after lightweighting according to an embodiment of the present invention;

[0059] Figure 11 This is a comparison curve of AUPRO in the image missing scenario before and after lightweighting according to an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, ordinary technicians in this field can also derive other embodiments without making any creative work.

[0061] To address the problems of large number of parameters, slow inference speed and limited edge resources in industrial device surface defect detection models, an embodiment of the present invention proposes a lightweight industrial device surface defect detection method that combines knowledge distillation and model pruning.

[0062] The method of the embodiment of the present invention first uses a large model (teacher network) to train multimodal normal samples. Then, the teacher knowledge is transferred to a lightweight student network through masked generative knowledge distillation, and then sorting channel pruning is used to further reduce redundant parameters, finally obtaining a lightweight defect detection model that can run in real time at the edge.

[0063] See also Figure 1 The embodiment of the present invention proposes a lightweight industrial device surface defect detection method. The core model is a lightweight defect detection framework that combines masking generation knowledge distillation and sorting pruning. Figure 1 The figure shows a schematic diagram of the overall lightweight defect detection framework provided by an embodiment of the present invention. The framework consists of four closely coordinated core modules:

[0064] (1) Teacher Network (T)

[0065] The teacher model is a multi-modal defect detection network based on a Transformer, which is cascaded by an image encoder, a point cloud encoder and a cross-modal attention fusion module. The image encoder adopts a Vision Transformer (ViT-Base) structure, divides the input image into fixed-size image blocks, and captures global texture features through a multi-layer self-attention mechanism. The point cloud encoder adopts a PointNet++ hierarchical structure, extracts geometric features of the point cloud through farthest point sampling and local geometric aggregation. The cross-modal attention fusion module: through a cross-attention mechanism, the image features and the point cloud features are deeply fused to output a unified multi-modal feature representation, which is used for subsequent defect detection.

[0066] (2) Student model (Student Network, S)

[0067] The student model is a dual-modal network after the teacher model is lightened, and the structure is as shown in Figure 2 The channel number is reduced to 50% of the teacher model, the Transformer layer number is compressed to 1 / 3 of the teacher model, and the student model includes a simplified image encoder, a point cloud encoder and a lightweight feature fusion module (1x1 convolution + channel attention). The student model additionally introduces a random masking layer in the training stage to implement mask generation knowledge distillation.

[0068] (3) Mask generation reconstruction module (Generator G)

[0069] The mask generation reconstruction module is composed of a 1x1 convolution layer, a 3x3 convolution layer and a ReLU activation function connected in sequence, and the function is to input the intermediate feature map randomly masked in the student model, and output the reconstructed features consistent with the feature dimension of the corresponding layer of the teacher model, which is used to calculate the distillation loss.

[0070] (4) Reorder pruning engine

[0071] The channel reservation order is optimized through a dynamic programming algorithm, and the flow is as shown in Figure 3 and Figure 4 The functions include graph modeling, maximum reward path search and channel rearrangement and pruning. Among them, the graph modeling divides the network into several segments according to the “producer-consumer” relationship, constructs an undirected graph with channels as nodes and shared relationships as edges. The maximum reward path search uses dynamic programming to solve the maximum reward value of the acyclic path, and the reward function considers the channel sharing degree and memory copy overhead. The channel rearrangement and pruning rearrange the convolution kernel and the feature map in the optimal path order, remove the redundant channels at a preset pruning rate, and finally restore the accuracy in the fine-tuning stage.

[0072] The lightened industrial device surface defect detection method of the embodiment of the application comprises the following steps:

[0073] (1) Teacher model training

[0074] First, high-resolution RGB images and corresponding 3D point cloud data of industrial device target objects (such as aircraft engine blades) under normal working conditions are collected synchronously at the industrial site. All samples are in a defect-free state, directly forming an unsupervised training set D containing only normal samples. train , no additional manual annotation is required.

[0075] Subsequently, the training set is used to train the teacher network T, a Transformer-based dual-modal defect detection network, end-to-end. Figure 5 As shown in the figure, the network first extracts global texture features from the 224×224×3 image through the Vision Transformer, and then uses the Point Transformer to perform farthest point sampling and local aggregation on the S-point point cloud to obtain the point cloud geometric features; after interpolation and projection, the two modalities are aligned to the same two-dimensional space. Then, as Figure 6 As shown in Figure 3, the cross-modal attention fusion module deeply fuses the two features into a unified multimodal representation. To compensate for the loss of fusion information, the teacher network establishes three additional storage repositories, one for the original image features, one for the original point cloud features, and one for the fused features. Finally, a single-class SVM is used to calculate the anomaly score. The training loss uses contrastive learning, which allows the network to learn a compact and generalizable normal mode distribution using only normal samples.

[0076] Finally, when the training loss converges, all weights θ of the teacher model T are T It is completely saved as a knowledge source and can be directly loaded and used in the subsequent masked generation knowledge distillation and sorting pruning stages.

[0077] (2) Masked Generative Knowledge Distillation

[0078] First, a lightweight student model S is constructed. Based on the teacher model T, the student model S reduces the number of channels by 50%, compresses the number of Transformer layers to 1 / 3, and reduces the overall parameter count from ≈180M to ≈90M, meeting the hard constraints of edge device memory <3GB and computing power <30TOPS.

[0079] Next, we enter the training phase. For each layer of the intermediate feature map F of the student model S S , a random binary mask M is generated according to the dynamic masking ratio, and the pixel-level masking sets the formula to cover the nth feature of the student by the nth random mask. The mask is resampled at each iteration to ensure that all pixels are covered in the full cycle, thereby improving the robustness of the student model to incomplete input.

[0080] Then, the masked features are restored through a lightweight reconstruction module, and the output dimension is consistent with the teacher features to achieve cross-dimensional alignment.

[0081] Then, calculate the distillation loss Loss diss . Reconstruction features and teacher characteristics F T The difference between them is measured using L2 distance or cosine similarity. The loss only acts on the masked area to avoid the student model from simply copying the teacher's features.

[0082] Finally, joint optimization is performed, and the total loss Loss all Loss ori is the original defect detection loss (reconstruction + density estimation). The SGD optimizer was used with a batch size of 256 and an initial learning rate of 0.1, which was decayed every 30 epochs until the validation set AUROC converged. After training, the student model S maintained detection accuracy while reducing inference time to 7.22ms per image and reducing video memory usage by 47%.

[0083] (3) Channel sorting pruning

[0084] First, the network is segmented, dividing the student model S into several segments based on the "producer-consumer" relationship. Each segment consists of a convolutional layer that generates a feature map (producer) and all subsequent convolutional layers that use this feature (consumers). This ensures that segments can be pruned independently without disrupting the overall network structure.

[0085] Graph modeling is then performed. For each segment, an undirected graph G = (V, E) is constructed, where a node v∈V represents the set of channels retained within the segment, and an edge e∈E indicates that two nodes share the same channel. If two channels are shared in subsequent layers, an edge is established between the corresponding nodes, with a weight equal to the number of shared channels.

[0086] The algorithm then searches for the maximum reward path, using dynamic programming to solve the "maximum reward acyclic path" problem. The reward matrix R records the number of retained channels at each node, while the edge reward matrix E records the number of shared channels. The maximum reward is recursively calculated to determine the order in which channels are retained. The algorithm first extracts the subgraph and then generates the final sorted sequence node by node according to a formula. This ensures that shared channels are centrally arranged after rearrangement, reducing memory duplication.

[0087] Channel rearrangement and pruning are then performed, rearranging the convolution kernels and feature maps according to the resulting order, and then removing redundant channels at a preset pruning rate. After pruning, the number of parameters is reduced from approximately 180M to 91M, and the inference time is reduced from 11.47ms to 7.22ms.

[0088] Finally, we fine-tune the pruned model S' for 5 epochs on the MVTec-3D AD dataset (SGD, lr = 0.01), reducing the accuracy loss to less than 2%. This results in a lightweight defect detection network that can run in real time at the edge.

[0089] (4) Defect detection

[0090] First, online inference is performed, and the samples to be inspected (RGB images or 3D point clouds) collected in real time on the production line are input into the pruned model S' deployed on the edge device (Jetson Nano / industrial PC). The single-frame forward latency is less than 10ms, and the video memory usage is less than 2.5GB.

[0091] Then, the anomaly score is generated and S' provides a dual-branch output. Among them, the reconstruction branch outputs the reconstructed image and reconstructed point cloud Calculate pixel-level / point-level mean squared error. The density estimation branch uses the normalization flow trained by the teacher network to map features to a standard Gaussian, resulting in a log-likelihood score. The results of the two branches are weighted and fused to form the final anomaly score A. The relevant parameters are automatically calibrated using a validation set.

[0092] Finally, perform defect determination and visualization, and set the threshold τ. If A>τ, the sample is determined to have defects, and a high-resolution residual heat map is immediately output, such as Figure 7 As shown, this system achieves pixel-level and point-level localization of defects such as scratches and holes, simultaneously triggering production line alarms and completing closed-loop quality inspection. Experiments show that S' maintains an Image-AUROC of 0.841 and an AUPRO of 0.933 on 10 object types in MVTec-3D AD, with an accuracy loss of less than 2%.

[0093] See also Figures 8 to 11 , which are the performance comparison curves of the embodiment of the present invention and multiple baseline methods in two incomplete modal scenarios: "point cloud missing" and "image missing". The horizontal axis in the figure is the 10 types of industrial objects (bagel, cable_gland, ..., tire) in the MVTec-3D AD dataset, and the vertical axis is the Image-AUROC ( Figure 8 and Figure 9 ) and AUPRO( Figure 10 and Figure 11 ), which is used to measure the comprehensive performance of defect detection and segmentation.

[0094] The curve “Teacher (Original)” represents the teacher model without any compression, and its performance is considered to be the theoretical upper limit.

[0095] The curve "Baseline-KD" represents a lightweight model that only uses traditional knowledge distillation and does not introduce masked generation and ranking pruning.

[0096] The curve "Baseline-Prune" represents a lightweight model that only uses conventional channel pruning without combining knowledge distillation.

[0097] The curve “KD+Reorder-50% (present invention)” represents the joint masked generative knowledge distillation and sorting channel pruning method proposed in the present invention, with the pruning rate set to 50%.

[0098] The comparison results show:

[0099] (a) Parameter size and inference speed: This paper compresses the model size from ≈180M to ≈90M, reduces the inference time from ≈12ms / sample to ≈7ms / sample, and reduces memory usage by more than 50%.

[0100] (b) Accuracy Preservation: Across all 10 categories, Image-AUROC only drops by about 1.5%, and AUPRO drops by ≤2%, significantly outperforming the accuracy losses of “Baseline-KD” and “Baseline-Prune”, which each have >3%.

[0101] (c) Robustness to missing modality: When the image or point cloud modality is missing, the AUROC of “Baseline-KD” and “Baseline-Prune” drops by 5% and 7%, respectively, while the curve of the present invention almost coincides with the “Teacher” curve, with a drop of <2%, indicating that masked generative distillation and ranking pruning jointly enhance the network’s robustness to incomplete input.

[0102] (d) Edge Deployability: On NVIDIA Jetson Nano, the proposed model achieves a single-frame latency of <10ms, meeting the real-time detection requirements of industrial sites.

[0103] In summary, the surface defect detection method for lightweight industrial devices in the embodiment of the present invention significantly reduces the model complexity and resource consumption while maintaining defect detection accuracy close to that of the teacher model, and exhibits excellent robustness and edge deployment friendliness in the modal missing scenario, fully verifying the effectiveness of the joint knowledge distillation and channel sorting pruning strategy.

[0104] The surface defect detection method for lightweight industrial devices according to the embodiment of the present invention is applicable to surface defect detection scenarios of industrial devices such as aircraft engine blades, 3C structural parts, and lithium electrode sheets, and can complete defect detection and positioning in real time at resource-constrained ends.

[0105] Another embodiment of the present invention further provides a lightweight industrial device surface defect detection system, comprising:

[0106] The teacher model training module is used to obtain either images or point cloud data of normal samples in industrial scenarios, or a combination of the two, to train the pre-established teacher model;

[0107] The student model training module is used to build a student model, transfer teacher features to the student network through masked generative knowledge distillation, and train the student model;

[0108] The pruned model acquisition module is used to perform channel sorting pruning on the trained student model to obtain a pruned model;

[0109] The defect judgment module is used to input the samples to be detected collected in real time into the pruning model to determine whether the samples to be detected have defects.

[0110] In one possible implementation, the teacher model training module is deployed on the cloud or a high-performance server. The Transformer-based dual-modal defect detection network is trained end-to-end using normal sample images and point cloud data collected at the industrial site, and the trained teacher model weights are saved as a knowledge source.

[0111] In one possible implementation, the student model training module runs on a GPU training node and has a built-in mask generation mechanism: during the training phase, the intermediate feature maps of the student model are randomly masked at a dynamic ratio of 10%–80%, and mapped to reconstructed features with the same dimension as the teacher features through a lightweight reconstruction module. The distillation loss is calculated using L2 distance or cosine similarity, and combined with the original defect detection loss to complete knowledge transfer.

[0112] In one possible implementation, the pruned model acquisition module divides the student model into several segments according to the "producer-consumer" relationship, constructs a channel-sharing undirected graph for each segment, uses the maximum reward path algorithm to determine the channel retention order, deletes redundant channels at a 50% pruning rate, and finally outputs the pruned lightweight model.

[0113] In one possible implementation, the defect determination module is deployed in an edge computing device or industrial PC, receives samples to be inspected in real time, calculates the reconstruction error or normalized flow probability density through a pruning model, obtains an anomaly score, and outputs a defect location heat map when the anomaly score exceeds a set threshold. The single-sample inference delay is less than 10ms.

[0114] In a possible implementation, a storage unit is further included, which is used to persist the teacher model weights, student model weights and pruned model weights, and supports model version management and online incremental updates.

[0115] Another embodiment of the present invention further provides an electronic device, including:

[0116] A memory storing at least one instruction; and a processor executing the instruction stored in the memory to implement the method for detecting surface defects of lightweight industrial devices.

[0117] Another embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores at least one instruction, and the at least one instruction is executed by a processor in an electronic device to implement the method for detecting surface defects of lightweight industrial devices.

[0118] Exemplarily, the instructions stored in the memory can be divided into one or more modules / units, which are stored in a computer-readable storage medium and executed by the processor to implement the lightweight industrial device surface defect detection method of the present invention. The one or more modules / units can be a series of computer-readable instruction segments capable of performing specific functions, which are used to describe the execution process of the computer program in the server.

[0119] The electronic device may be a computing device such as a smartphone, laptop, PDA, or cloud server. The electronic device may include, but is not limited to, a processor and memory. Those skilled in the art will appreciate that the electronic device may include more or fewer components, or a combination of certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, and the like.

[0120] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0121] The memory may be an internal storage unit of the server, such as a hard disk or memory of the server. The memory may also be an external storage device of the server, such as a plug-in hard disk equipped on the server, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Furthermore, the memory may include both an internal storage unit of the server and an external storage device. The memory is used to store the computer-readable instructions and other programs and data required by the server. The memory may also be used to temporarily store data that has been output or is about to be output.

[0122] It should be noted that the information interaction, execution process, etc. between the above-mentioned module units are based on the same concept as the method embodiment. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0123] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0124] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the camera / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunications signal and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk.

[0125] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0126] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A method for detecting surface defects of lightweight industrial devices, characterized in that: include: Obtain either or a combination of image and point cloud data of normal samples in industrial scenes to train the pre-established teacher model; Build a student model, transfer teacher features to the student network through masked generative knowledge distillation, and train the student model; Perform channel sorting and pruning on the trained student model to obtain a pruned model; The samples to be tested collected in real time are input into the pruning model to determine whether the samples to be tested have defects.

2. The method for detecting surface defects of lightweight industrial devices according to claim 1, characterized in that: The teacher model is built using a Transformer-based multimodal defect detection network, which is composed of a cascade of image encoder, point cloud encoder and cross-modal attention fusion module; The image encoder adopts the Vision Transformer structure to divide the input image into image blocks of fixed size and capture global texture features through a multi-layer self-attention mechanism; The point cloud encoder adopts the deep point cloud hierarchical feature learning network PointNet++ hierarchical structure to extract the geometric features of the point cloud through farthest point sampling and local geometric aggregation; The cross-modal attention fusion module deeply fuses image features with point cloud features through a cross-attention mechanism, and outputs a unified multimodal feature representation for subsequent defect detection.

3. The method for detecting surface defects of lightweight industrial devices according to claim 1, characterized in that: The student model is established by using a network structure that is a lightweight version of the teacher model, and the number of network layers and channels are smaller than the teacher model; During the training phase, the intermediate feature map input to the student model is randomly masked with pixels according to a preset masking ratio to obtain masked features; Map the masked features to reconstructed features consistent with the feature dimensions of the corresponding layer of the teacher model; Calculate the distillation loss Loss between the reconstructed features and the teacher features diss ; Joint original defect detection loss Loss ori Distillation loss diss Train the student model.

4. The method for detecting surface defects of lightweight industrial devices according to claim 3, characterized in that: In the step of performing random pixel masking on the intermediate feature map input to the student model according to a preset masking ratio, the masking ratio is a fixed value or a dynamic value between 10% and 80%.

5. The method for detecting surface defects of lightweight industrial devices according to claim 3, characterized in that: The masked features are mapped to reconstructed features with the same feature dimensions as the corresponding layer of the teacher model through a masked generation and reconstruction module, which consists of a sequentially connected 1×1 convolutional layer, a 3×3 convolutional layer, and a ReLU activation function.

6. The method for detecting surface defects of lightweight industrial devices according to claim 3, characterized in that: The distillation loss between the reconstructed features and the teacher features is Loss diss L2 distance or cosine similarity is used for measurement.

7. The method for detecting surface defects of lightweight industrial devices according to claim 1, characterized in that: The step of performing channel sorting and pruning on the trained student model to obtain a pruned model includes: Divide the student model's network into several segments based on producer-consumer relationships; Construct an undirected graph for each network segment, where nodes represent channel sets and edges represent channel sharing relationships; The maximum reward path algorithm is used to determine the channel retention order. The maximum reward path algorithm is implemented by solving the maximum reward value of the acyclic path through dynamic programming, and the convolution kernels and feature maps are rearranged in the corresponding order. Delete redundant channels at a preset pruning rate to obtain a pruned model; In the step of inputting the real-time sample to be detected into the pruning model and determining whether the sample to be detected has defects, a reconstruction error or a density estimation anomaly score of the sample to be detected is calculated, where the anomaly score is obtained by calculating the mean square error between the input image and the reconstructed image or a probability density based on a normalized flow; When the anomaly score is greater than the threshold, it is determined that the sample to be tested has defects.

8. A lightweight industrial device surface defect detection system, characterized in that: include: The teacher model training module is used to obtain either images or point cloud data of normal samples in industrial scenarios, or a combination of the two, to train the pre-established teacher model; The student model training module is used to build a student model, transfer teacher features to the student network through masked generative knowledge distillation, and train the student model; The pruned model acquisition module is used to perform channel sorting pruning on the trained student model to obtain a pruned model; The defect judgment module is used to input the samples to be detected collected in real time into the pruning model to determine whether the samples to be detected have defects.

9. An electronic device, characterized in that: include: a memory storing at least one instruction; and A processor executes instructions stored in the memory to implement the surface defect detection method for lightweight industrial devices according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, and the at least one instruction is executed by a processor in an electronic device to implement the surface defect detection method for lightweight industrial devices according to any one of claims 1 to 7.

Images