Defect detection method for complex texture surface and related device

Abstract

The invention discloses a defect detection method for a complex texture surface and a related device, and relates to the technical field of machine vision, and the method comprises the steps: carrying out the preprocessing of a surface image of a complex texture product; slicing the surface image to obtain a plurality of sub-image blocks; dividing the plurality of sub-image blocks into a plurality of processing image block groups and distributing the processing image block groups to corresponding AI processor cores; each AI processor core adopts a defect detection model based on CANN optimization to carry out defect analysis on each sub-image block in the processed image block group to obtain corresponding defect information; performing fusion processing on the defect information of each sub-image block based on a non-maximum suppression algorithm to obtain global defect information; and performing defect grading judgment based on the global defect information so as to perform product decision execution and model optimization. According to the invention, the precision and efficiency of defect detection are improved, and the problem of insufficient real-time computing power of the current processor is solved.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and in particular to a method and related apparatus for detecting defects on complex textured surfaces. Background Technology

[0002] In modern industrial manufacturing, ensuring flawless product surfaces is crucial for guaranteeing final quality. Initially, the industry relied on manual visual inspection to achieve this goal. However, this method is subjective, inefficient, and prone to missed or false positives due to human fatigue, making it unsuitable for automated, high-speed production lines. To address this, machine vision technology based on image processing algorithms emerged. While this technology performs reasonably well with products on simple backgrounds and smooth surfaces, it struggles to effectively distinguish between normal texture variations and genuine minute defects when applied to surfaces with complex textures, resulting in high false positive and false negative rates. With the development of deep learning technology, AI-based detection methods have achieved breakthroughs in feature recognition capabilities by learning from massive amounts of data, significantly improving their ability to understand complex textures.

[0003] Nevertheless, existing general-purpose detection solutions also face new technical bottlenecks. On the one hand, accurately identifying minute defects hidden beneath the texture of high-resolution images requires running extremely complex deep neural network models. This places a huge real-time computing burden on a single or few processing units, making it difficult to simultaneously meet the requirements of high accuracy and low latency. On the other hand, training complex models is not only time-consuming, but also the iteration and redeployment processes cannot quickly respond to the needs of flexible production when production lines change or new defects emerge. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art. This invention provides a method and related device for detecting defects on complex textured surfaces, which improves the accuracy and efficiency of defect detection and solves the problem of insufficient real-time computing power of current processors.

[0005] To address the aforementioned technical problems, this invention provides a method for detecting defects on complex textured surfaces, the method comprising: The surface image of the complex textured product is acquired using the image acquisition module, and the surface image is preprocessed to obtain a preprocessed surface image. The preprocessed surface image is sliced ​​to obtain several sub-image blocks; Several sub-image blocks are divided into several processing image block groups, and each processing image block group is assigned to the corresponding artificial intelligence (AI) processor core; Each AI processor core uses a defect detection model optimized based on the neural network heterogeneous computing architecture CANN to perform defect analysis on each sub-image block in the processed image block group and obtain the corresponding defect information; The defect information of each sub-image block is fused based on the non-maximum suppression algorithm to obtain global defect information; Defect classification is performed based on global defect information to obtain defect classification results, and product decision execution and model optimization are performed based on the defect classification results.

[0006] Optionally, the preprocessing of the surface image to obtain a preprocessed surface image includes: The surface image is subjected to image enhancement processing to obtain an enhanced surface image; The surface image after image enhancement is normalized to obtain a normalized surface image; The normalized surface image is resized to obtain a preprocessed surface image.

[0007] Optionally, the step of slicing the preprocessed surface image to obtain several sub-image blocks includes: Obtain the target input size of the model and the number of computing nodes in the AI ​​cluster; Based on the target input size of the model and the number of computing nodes, the preprocessed surface image is sliced ​​to obtain several sub-image blocks.

[0008] Optionally, the step of dividing several sub-image blocks into several processing image block groups and assigning each processing image block group to a corresponding artificial intelligence (AI) processor core includes: Identify several AI processor cores that are in an idle state, and divide several sub-image blocks into several processing image block groups based on the number of AI processor cores in an idle state; The resource scheduling system allocates each image block group to the corresponding AI processor cores that are in a state.

[0009] Optionally, each AI processor core employs a defect detection model optimized based on the CANN (Neural Network Heterogeneous Computing Architecture) to perform defect analysis on each sub-image block within the processed image block group, obtaining corresponding defect information, including: Collect a set of product sample images and label the defect locations and defect types in the set of product sample images to obtain a labeled set of product sample images; A convolutional neural network model is trained using a defect sensitivity loss function based on a set of labeled product sample images. The convolutional neural network includes a DarkNet53 backbone network, a neck network for multi-scale feature fusion, and a multi-class regression network. Based on CANN, the trained convolutional neural network model is compiled and optimized to obtain a defect detection model, which is then deployed to the corresponding AI processor cores. Each sub-image block in the processed image block group is input into the defect detection model of each AI processor core for defect category, defect location and confidence score analysis, to obtain the detection box and confidence score of each sub-image block, and to generate defect information based on the detection box and confidence score.

[0010] Optionally, the step of fusing the defect information of each sub-image block based on the non-maximum suppression algorithm to obtain global defect information includes: Obtain the overlapping area between sub-image blocks and determine the detection box in each defect information of the overlapping area; The confidence scores of each detection box in the overlapping region are weighted and fused based on the attention algorithm to obtain the first confidence score of each detection box; Obtain the visibility ratio of each detection box in the overlapping region, and determine the non-maximum suppression threshold using a non-maximum suppression threshold analysis model based on the first confidence score and visibility ratio of each detection box in the overlapping region. The target detection box in the overlapping region is determined based on the first confidence score and the non-maximum suppression threshold. Global defect information is obtained by performing position mapping and information fusion processing based on the target detection bounding boxes in the overlapping regions and the defect information of each sub-image block.

[0011] Optionally, the weighted fusion processing of the confidence scores of each detection box in the overlapping region based on the attention algorithm to obtain the first confidence score of each detection box includes: Determine the complete intersection-union loss function for each detection box in the overlapping region; The attention weights for the confidence score are determined based on the full intersection-union loss function, and the attention weights and the confidence score are weighted and fused to obtain the first confidence score.

[0012] Optionally, the step of performing defect classification judgment based on global defect information, obtaining defect classification judgment results, and performing product decision execution and model optimization based on the defect classification judgment results includes: Based on preset judgment rules, the global defect information is used to perform defect classification judgment on complex texture products, and the defect classification judgment result is obtained. Based on the defect classification judgment result, a control instruction is generated and sent to the programmable logic controller (PLC) of the production line. The PLC controls the corresponding equipment on the production line to execute the control instruction. Based on the defect classification judgment result, it is determined whether model optimization is needed. If it is determined that model optimization is needed, the defect classification judgment result is transmitted to the offline training module, and the defect detection model is learned and optimized based on the offline training model using the defect classification judgment result.

[0013] In addition, the present invention also provides a defect detection device for complex textured surfaces, the device comprising: Image preprocessing module: used to acquire surface images of complex textured products based on the image acquisition module, and to preprocess the surface images to obtain preprocessed surface images; Image slicing module: used to slice the preprocessed surface image to obtain several sub-image blocks; Image block allocation module: used to divide several sub-image blocks into several processing image block groups, and allocate each processing image block group to the corresponding artificial intelligence (AI) processor core; Defect Analysis Module: This module is used by each AI processor core to perform defect analysis on each sub-image block in the processed image block group using a defect detection model optimized based on the neural network heterogeneous computing architecture CANN, and to obtain the corresponding defect information. Information fusion module: used to fuse the defect information of each sub-image block based on the non-maximum suppression algorithm to obtain global defect information; Defect Decision Module: Used to classify and judge defects based on global defect information, obtain defect classification and judgment results, and perform product decision execution and model optimization based on the defect classification and judgment results.

[0014] In addition, the present invention also provides a computer-readable storage medium that stores computer instructions that, when executed on an electronic device, cause the electronic device to perform the above-described method for detecting defects on complex textured surfaces.

[0015] In this embodiment of the invention, the preprocessed surface image is sliced ​​to obtain several sub-image blocks. These sub-image blocks are then divided into several processing image block groups. Each processing image block group is assigned to a corresponding AI processor core. Each AI processor core uses a CANN-optimized defect detection model to analyze defects in each sub-image block within its processing image block group. A non-maximum suppression algorithm is used to fuse the defect information of each sub-image block to obtain global defect information. Each processor core performs defect analysis concurrently, transforming a serial computation task into a parallel task. This solves the problem of insufficient real-time computing power preventing the deployment of high-precision, complex models, thus improving defect detection efficiency. Simultaneously, the CANN-optimized defect detection model improves defect detection accuracy, avoiding misjudging normal texture fluctuations as defects. Defect classification is performed based on the global defect information to obtain defect classification results. Based on these results, product decision execution and model optimization are performed, achieving integrated model training and inference, and enabling model self-optimization. This adapts to the requirements of flexible production and rapid response in high-end manufacturing. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the defect detection method for complex textured surfaces in an embodiment of the present invention; Figure 2 This is a flowchart illustrating a defect detection method for complex textured surfaces according to another embodiment of the present invention. Figure 3 This is a schematic diagram of the structural composition of the defect detection device for complex textured surfaces in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Example 1 Please see Figure 1 , Figure 1This is a flowchart illustrating a defect detection method for complex textured surfaces according to an embodiment of the present invention. The method includes: S11: Acquire a surface image of a complex textured product based on the image acquisition module, and preprocess the surface image to obtain a preprocessed surface image; In the specific implementation of this invention, a surface image of a complex textured product is acquired based on an image acquisition module. The surface image is then enhanced to obtain an enhanced surface image. The enhanced surface image is then normalized to obtain a normalized surface image. Finally, the normalized surface image is resized to make it clearer and facilitate subsequent image slicing.

[0020] S12: Slice the preprocessed surface image to obtain several sub-image blocks; In the specific implementation of this invention, the target input size of the model and the number of computing nodes of the Artificial Intelligence (AI) cluster are obtained; based on the target input size of the model and the number of computing nodes, the preprocessed surface image is sliced ​​to improve the speed of image defect analysis.

[0021] S13: Divide several sub-image blocks into several processing image block groups, and assign each processing image block group to the corresponding artificial intelligence (AI) processor core; In the specific implementation of this invention, several AI processor cores in an idle state are identified, and several sub-image blocks are divided into several processing image block groups based on the number of AI processor cores in an idle state. Based on the resource scheduling system, each processing image block group is allocated to the corresponding AI processor cores in an idle state, so that the corresponding AI processor core processes its own corresponding sub-image block, thereby enabling parallel reasoning of image defects.

[0022] S14: Each AI processor core uses a defect detection model optimized based on the neural network heterogeneous computing architecture CANN to perform defect analysis on each sub-image block in the processed image block group and obtain the corresponding defect information; In the specific implementation of this invention, defect locations and defect categories are labeled in a set of product sample images to obtain a labeled set of product sample images. Based on the labeled set of product sample images, a convolutional neural network model is trained using a defect sensitivity loss function. The convolutional neural network includes a DarkNet53 backbone network, a multi-scale feature fusion neck network, and a multi-classification regression network. The trained convolutional neural network model is compiled and optimized using a Compute Architecture for Neural Networks (CANN) to obtain a defect detection model, which is then deployed to the corresponding AI processor cores. Each sub-image block in the processed image block group is input into the defect detection model of each AI processor core for defect category, defect location, and confidence score analysis to obtain the detection box and confidence score of each sub-image block. Defect information is generated based on the detection box and confidence score. Each AI processor core performs concurrent inference, which can complete the defect analysis of the sub-image blocks in a very short time, meeting the requirement that the total system latency is far lower than the cycle time of a high-speed production line.

[0023] S15: Based on the non-maximum suppression algorithm, the defect information of each sub-image block is fused to obtain global defect information; In the specific implementation of this invention, the overlapping region between sub-image blocks is obtained, and the detection boxes in each defect information of the overlapping region are determined; the confidence scores of each detection box in the overlapping region are weighted and fused based on the attention algorithm to obtain the first confidence score of each detection box; the visibility ratio of each detection box in the overlapping region is obtained, and the non-maximum suppression threshold is determined using a non-maximum suppression threshold analysis model based on the first confidence score and visibility ratio of each detection box in the overlapping region; the target detection box in the overlapping region is determined based on the first confidence score and the non-maximum suppression threshold; the position mapping and information fusion processing are performed based on the target detection box in the overlapping region and the defect information of each sub-image block to obtain global defect information, which solves the problem of false positives and false negatives caused by insufficient model capabilities in traditional methods, and improves the defect detection accuracy to a new level.

[0024] S16: Based on global defect information, perform defect classification judgment to obtain defect classification judgment results, and perform product decision execution and model optimization based on the defect classification judgment results.

[0025] In the specific implementation of this invention, based on preset judgment rules, global defect information is used to classify and judge defects in complex textured products, and the defect classification judgment result is obtained. Based on the defect classification judgment result, control instructions are generated and sent to the programmable logic controller (PLC) of the production line. The PLC controls the corresponding equipment on the production line to execute the control instructions. Based on the defect classification judgment result, it is determined whether model optimization is required. If it is determined that model optimization is required, the defect classification judgment result is transmitted to the offline training module. Based on the offline training model, the defect detection model is learned and optimized using the defect classification judgment result, which shortens the model retraining cycle and perfectly adapts to the requirements of high-end manufacturing for flexible production and rapid response.

[0026] In this embodiment of the invention, the preprocessed surface image is sliced ​​to obtain several sub-image blocks. These sub-image blocks are then divided into several processing image block groups. Each processing image block group is assigned to a corresponding AI processor core. Each AI processor core uses a CANN-optimized defect detection model to analyze defects in each sub-image block within its processing image block group. A non-maximum suppression algorithm is used to fuse the defect information of each sub-image block to obtain global defect information. Each processor core performs defect analysis concurrently, transforming a serial computation task into a parallel task. This solves the problem of insufficient real-time computing power preventing the deployment of high-precision, complex models, thus improving defect detection efficiency. Simultaneously, the CANN-optimized defect detection model improves defect detection accuracy, avoiding misjudging normal texture fluctuations as defects. Defect classification is performed based on the global defect information to obtain defect classification results. Based on these results, product decision execution and model optimization are performed, achieving integrated model training and inference, and enabling model self-optimization. This adapts to the requirements of flexible production and rapid response in high-end manufacturing.

[0027] Example 2 Please see Figure 2 , Figure 2 This is a flowchart illustrating a defect detection method for complex textured surfaces according to another embodiment of the present invention, the method comprising: S201: Acquire a surface image of a complex textured product based on the image acquisition module, and preprocess the surface image to obtain a preprocessed surface image; In a specific implementation of the present invention, the preprocessing of the surface image to obtain a preprocessed surface image includes: performing image enhancement processing on the surface image to obtain an image-enhanced surface image; performing normalization processing on the image-enhanced surface image to obtain a normalized surface image; and performing size adjustment processing on the normalized surface image to obtain a preprocessed surface image.

[0028] Specifically, the image acquisition module acquires surface images of products with complex textures. The image acquisition module is deployed above the production line and consists of one or more industrial cameras with more than 50 million pixels and matching coaxial or ring light sources. Taking Mini LED automotive light chips as an example, this module is responsible for capturing images of the complete chip array containing tens of thousands of LED chips in an instant as the chips flow through the inspection station, ensuring that the details of each chip are clearly visible.

[0029] The surface image undergoes image enhancement processing, including contrast adjustment, color enhancement, sharpening, and noise reduction, to obtain an enhanced surface image. The enhanced surface image is then normalized, including coordinate centering, x-shearing normalization, scaling normalization, and rotation normalization, to obtain a normalized surface image. Finally, the normalized surface image is resized to a suitable size, avoiding images that are too large or too small, resulting in a preprocessed surface image.

[0030] S202: Slice the preprocessed surface image to obtain several sub-image blocks; In the specific implementation of this invention, the step of slicing the preprocessed surface image to obtain several sub-image blocks includes: obtaining the target input size of the model and the number of computing nodes in the AI ​​cluster; and slicing the preprocessed surface image based on the target input size of the model and the number of computing nodes to obtain several sub-image blocks.

[0031] Specifically, the target input size of the model and the number of computing nodes in the AI ​​cluster are obtained. The target input size of the model is the optimal input size for the model. The AI ​​cluster is the Ascend AI computing cluster, which internally interconnects multiple Ascend AI processors via a high-speed RoCE network. Based on the target input size of the model and the number of computing nodes, the preprocessed surface image is sliced ​​to obtain several sub-image blocks. According to the number of computing nodes in the cluster and the optimal input size of the model, the surface image is automatically divided into several sub-image blocks with overlapping regions. The overlapping regions are designed to ensure that defects at the edges of the image blocks can be completely detected.

[0032] S203: Divide several sub-image blocks into several processing image block groups, and assign each processing image block group to the corresponding AI processor core; In the specific implementation of this invention, the step of dividing several sub-image blocks into several processing image block groups and allocating each processing image block group to a corresponding artificial intelligence (AI) processor core includes: determining several AI processor cores that are in an idle state; dividing several sub-image blocks into several processing image block groups based on the number of AI processor cores in an idle state; and allocating each processing image block group to a corresponding AI processor core in an idle state based on a resource scheduling system.

[0033] Specifically, a number of AI processor cores in an idle state are identified. Based on the number of idle AI processor cores, several sub-image blocks are divided into several processing image block groups. That is, all sub-image blocks are divided into several groups according to the number of processor cores, and each group corresponds to one processor core. The resource scheduling system based on the Ascend cluster allocates each processing image block group to the corresponding idle AI processor cores. Each AI processor core is only responsible for processing one or several sub-image blocks assigned to its own processing image block group.

[0034] S204: Collect a set of product sample images, and annotate the defect locations and defect types in the set of product sample images to obtain an annotated set of product sample images; In the specific implementation of this invention, a set of product sample images is collected, and the defect locations and defect categories are marked on the set of product sample images. That is, the location and category of the defects are marked on the set of product sample images. The defect categories include grain loss, abnormal brightness, positional displacement, surface scratches, foreign matter contamination, etc., to obtain the marked set of product sample images.

[0035] S205: A convolutional neural network model is trained using a defect sensitivity loss function based on a set of labeled product sample images. The convolutional neural network includes a DarkNet53 backbone network, a neck network for multi-scale feature fusion, and a multi-classification regression network. In the specific implementation of this invention, a convolutional neural network model is trained using a defect sensitivity loss function based on a set of labeled product sample images. The convolutional neural network includes a DarkNet53 backbone network, a multi-scale feature fusion neck network, and a multi-class regression network. The defect sensitivity loss function includes a loss term based on region attention, a loss term based on edge feature preservation, and a loss term based on class balance. Applying the loss term based on region attention during model training improves the model's ability to detect small-sized defect regions. Applying the loss term based on edge feature preservation enhances the accuracy of defect boundary localization. Applying the loss term based on class balance mitigates the impact of imbalanced sample numbers for different defect types on detection performance. The DarkNet53 backbone network includes several individual convolutional layers and a combined convolutional layer. The combined convolutional layer includes several downsampling convolutional layers, residual blocks, and a spatial attention module. The multi-scale feature fusion neck network includes several max-pooling layers, convolutional layers, and a feature fusion convolutional network. The multi-class regression network includes several convolutional layers.

[0036] S206: Based on CANN, the trained convolutional neural network model is compiled and optimized to obtain a defect detection model, and the defect detection model is deployed to the corresponding AI processor cores; In the specific implementation of this invention, the trained convolutional neural network model is compiled and optimized using the corresponding debugging and optimization tools of CANN to obtain a defect detection model. The defect detection model is then deployed to the corresponding AI processor cores. CANN is an architecture specifically designed and optimized for high-performance neural network computing needs. At the hardware level, the Ascend AI processor features customized computing resources and deep adaptation in its functional implementation, providing a strong hardware foundation for improving the computing performance of neural networks. At the software level, the software stack included in CANN provides functions for managing network models, computational flows, and data flows, supporting the execution flow of neural networks on heterogeneous processors.

[0037] S207: Input each sub-image block in the processed image block group into the defect detection model of each AI processor core to analyze the defect category, defect location and confidence score, obtain the detection box and confidence score of each sub-image block, and generate defect information based on the detection box and confidence score; In the specific implementation of this invention, each sub-image block in the processed image block group is input into the defect detection model of each AI processor core for defect category, defect location, and confidence score analysis. This yields the detection box and confidence score for each sub-image block. The detection box includes information on the defect category and defect location of each sub-image block, and defect information is generated based on the detection box and confidence score. Leveraging the distributed computing power and memory resources of the Ascend AI cluster, it is possible to deploy and run ultra-large-scale deep learning models with billions of parameters. These models can learn the fine design paradigms and generation rules behind complex textures from massive amounts of data, thus possessing capabilities ranging from pattern recognition to anomaly understanding. This capability enables the accurate identification of any micron-level defects deviating from normal paradigms, fundamentally solving the problem of false positives and false negatives caused by insufficient model capabilities in traditional methods, and elevating detection accuracy to a new level. By slicing large images and distributing them to a massive number of AI cores in the cluster for simultaneous processing, the originally time-consuming serial task is transformed into an extremely short parallel task. For example, a single core can process a sub-image tile in just 10 milliseconds. After the overhead of distribution and aggregation, the total time for the entire cluster to process a complete large image can still be controlled within 50 milliseconds, which is far below the production line cycle time requirement, thus perfectly solving the real-time computing power bottleneck.

[0038] S208: Based on the non-maximum suppression algorithm, the defect information of each sub-image block is fused to obtain global defect information; In the specific implementation of this invention, the step of fusing the defect information of each sub-image block based on the non-maximum suppression algorithm to obtain global defect information includes: obtaining the overlapping region between sub-image blocks and determining the detection boxes in each defect information of the overlapping region; performing weighted fusion processing on the confidence scores of each detection box in the overlapping region based on the attention algorithm to obtain the first confidence score of each detection box; obtaining the visibility ratio of each detection box in the overlapping region, and determining the non-maximum suppression threshold based on the first confidence score and visibility ratio of each detection box in the overlapping region using the non-maximum suppression threshold analysis model; determining the target detection box in the overlapping region based on the first confidence score and the non-maximum suppression threshold; and performing position mapping and information fusion processing based on the target detection box in the overlapping region and the defect information of each sub-image block to obtain global defect information.

[0039] Specifically, the overlapping regions between sub-image blocks are obtained, and the detection boxes in each defect information of the overlapping regions are determined, that is, the detection boxes of each overlapping region are extracted. The confidence scores of each detection box in the overlapping regions are weighted and fused based on an attention algorithm. The attention weight of the confidence score is determined through the attention algorithm, and the first confidence score of each detection box is obtained by weighted fusion based on the attention weight and the confidence score.

[0040] The visibility ratio of each detection box in the overlapping region is obtained. The visibility ratio is the area ratio between the visible border of the detection box and the total border. Based on the first confidence score and visibility ratio of each detection box in the overlapping region, the non-maximum suppression threshold is determined using a non-maximum suppression threshold analysis model. The non-maximum suppression threshold analysis model is a converged model obtained by inputting the sample dataset into a deep neural network for training. The model includes several fully connected layers and activation function layers. The fully connected layers are used to obtain the confidence score and visibility ratio of the detection boxes, and the activation function layers are used to output the non-maximum suppression threshold.

[0041] The target detection box in the overlapping region is determined based on the first confidence score and the non-maximum suppression threshold. The intersection-union ratio (IUR) of the current detection box in the overlapping region with the other detection boxes is calculated. If the IUR is greater than the non-maximum suppression threshold, the detection box is suppressed. If the IUR is less than or equal to the non-maximum suppression threshold, the detection box is retained. Among the retained detection boxes, the detection box with the highest first confidence score is selected as the target detection box.

[0042] The system performs position mapping and information fusion processing based on the target detection bounding boxes in overlapping regions and the defect information of each sub-image patch. Specifically, it extracts the defect information from the target detection bounding boxes in the overlapping regions and performs position mapping and information fusion with the defect information of each sub-image patch (excluding the overlapping regions). These local results based on the sub-image patch coordinate system are then accurately remapped back to the global coordinate system of the complete large image according to the original slice positions. Multiple detection results in the overlapping regions are then fused to obtain global defect information, which can be a complete global result map marking all potential defects. This mechanism of aggregating and remapping all local results to generate the global detection map transforms the originally second-level serial computation task into a millisecond-level parallel task, ensuring that the total system latency is far lower than the cycle time requirements of high-speed production lines.

[0043] Furthermore, the step of performing weighted fusion processing on the confidence scores of each detection box in the overlapping region based on the attention algorithm to obtain the first confidence score of each detection box includes: determining the full intersection-union loss function of each detection box in the overlapping region; determining the attention weight of the confidence score based on the full intersection-union loss function; and performing weighted fusion of the attention weight and the confidence score to obtain the first confidence score.

[0044] Specifically, the Full Cross-Union Ratio (MCR) loss function is determined for each detection box in the overlapping region. The MCR loss function is a bounding box regression loss function used in object detection tasks. It further considers the aspect ratio effect on top of the Distance MCR loss function, making the object box regression more stable and accurate. Based on the MCR loss function, attention weights for the confidence score are determined. A detection box is selected from the detection boxes in each overlapping region. The MCR loss functions of the selected detection box and the remaining detection boxes are normalized to obtain attention weights. These attention weights and the confidence score are then weighted and fused to obtain a first confidence score, which improves the accuracy of the detection box confidence score.

[0045] S209: Based on global defect information, perform defect classification judgment, obtain defect classification judgment results, and perform product decision execution and model optimization based on the defect classification judgment results.

[0046] In a specific implementation of this invention, the step of performing defect classification judgment based on global defect information, obtaining defect classification judgment results, and performing product decision execution and model optimization based on the defect classification judgment results includes: performing defect classification judgment on complex texture products using the global defect information based on preset judgment rules, and obtaining defect classification judgment results; generating control instructions based on the defect classification judgment results, and sending the control instructions to the programmable logic controller (PLC) of the production line, wherein the PLC controls the corresponding equipment on the production line to execute the control instructions; determining whether model optimization is required based on the defect classification judgment results, and if it is determined that model optimization is required, transmitting the defect classification judgment results to the offline training module, and using the defect classification judgment results to learn and optimize the defect detection model based on the offline training model.

[0047] Specifically, based on preset judgment rules, the global defect information is used to classify and judge defects in complex texture products, and the defect classification judgment results are obtained. The judgment is made according to the rules preset in the quality control system. For example, the rules can be set as follows: grain loss with a confidence level higher than 90% is considered a serious defect; scratches with a length of more than 50 micrometers are considered serious defects; and brightness abnormalities with a confidence level between 80% and 90% may be marked as requiring re-inspection. At the same time, when a new type of defect appears, it will also be marked as a difficult sample for feedback.

[0048] Based on the defect classification judgment result, a control instruction is generated. If the defect classification judgment result determines that it is a serious defect, a product removal control instruction is generated and sent to the programmable logic controller (PLC) of the production line. The PLC controls the corresponding equipment on the production line to execute the control instruction. When the product removal control instruction is received, the PLC controls the drive rejection device of the production line to accurately remove the corresponding product when it arrives at the designated workstation. The entire process is completed within milliseconds.

[0049] Based on the defect classification results, it is determined whether model optimization is needed. If model optimization is deemed necessary (i.e., the defect classification results are marked as challenging feedback samples), optimization is required. The defect classification results are then transmitted to the offline training module. The offline training model uses these results to learn and optimize the defect detection model. The cluster uses this data to retrain the model, enabling it to learn to identify these difficult cases and generate a more intelligent new model. Leveraging the cluster's powerful distributed training capabilities, the system completes model retraining and optimization within hours, shortening the previous model iteration cycle of several weeks by several orders of magnitude. This rapid response and continuous self-optimization capability perfectly meets the urgent needs of modern manufacturing for flexible and intelligent production.

[0050] In this embodiment of the invention, the preprocessed surface image is sliced ​​to obtain several sub-image blocks. These sub-image blocks are then divided into several processing image block groups. Each processing image block group is assigned to a corresponding AI processor core. Each AI processor core uses a CANN-optimized defect detection model to analyze defects in each sub-image block within its processing image block group. A non-maximum suppression algorithm is used to fuse the defect information of each sub-image block to obtain global defect information. Each processor core performs defect analysis concurrently, transforming a serial computation task into a parallel task. This solves the problem of insufficient real-time computing power preventing the deployment of high-precision, complex models, thus improving defect detection efficiency. Simultaneously, the CANN-optimized defect detection model improves defect detection accuracy, avoiding misjudging normal texture fluctuations as defects. Defect classification is performed based on the global defect information to obtain defect classification results. Based on these results, product decision execution and model optimization are performed, achieving integrated model training and inference, and enabling model self-optimization. This adapts to the requirements of flexible production and rapid response in high-end manufacturing.

[0051] Example 3 Please see Figure 3 , Figure 3 This is a schematic diagram of the structural composition of a defect detection device for complex textured surfaces according to an embodiment of the present invention. The device includes: Image preprocessing module 31: used to acquire a surface image of a complex textured product based on the image acquisition module, and to preprocess the surface image to obtain a preprocessed surface image; Image slicing module 32: used to slice the preprocessed surface image to obtain several sub-image blocks; Image block allocation module 33: used to divide several sub-image blocks into several processing image block groups, and allocate each processing image block group to the corresponding artificial intelligence (AI) processor core; Defect Analysis Module 34: This module is used by each AI processor core to perform defect analysis on each sub-image block in the processed image block group using a defect detection model optimized based on the neural network heterogeneous computing architecture CANN, and to obtain the corresponding defect information. Information fusion module 35: used to fuse the defect information of each sub-image block based on the non-maximum suppression algorithm to obtain global defect information; Defect Decision Module 36: Used to perform defect classification judgment based on global defect information, obtain defect classification judgment results, and perform product decision execution and model optimization based on the defect classification judgment results.

[0052] In the specific implementation of this invention, the specific implementation of the device item can be referred to the implementation of the method item above, and will not be repeated here.

[0053] In this embodiment of the invention, the preprocessed surface image is sliced ​​to obtain several sub-image blocks. These sub-image blocks are then divided into several processing image block groups. Each processing image block group is assigned to a corresponding AI processor core. Each AI processor core uses a CANN-optimized defect detection model to analyze defects in each sub-image block within its processing image block group. A non-maximum suppression algorithm is used to fuse the defect information of each sub-image block to obtain global defect information. Each processor core performs defect analysis concurrently, transforming a serial computation task into a parallel task. This solves the problem of insufficient real-time computing power preventing the deployment of high-precision, complex models, thus improving defect detection efficiency. Simultaneously, the CANN-optimized defect detection model improves defect detection accuracy, avoiding misjudging normal texture fluctuations as defects. Defect classification is performed based on the global defect information to obtain defect classification results. Based on these results, product decision execution and model optimization are performed, achieving integrated model training and inference, and enabling model self-optimization. This adapts to the requirements of flexible production and rapid response in high-end manufacturing.

[0054] This invention provides a computer-readable storage medium storing a computer program. When executed by a processor, this program implements the defect detection method for complex textured surfaces according to any of the above embodiments. The computer-readable storage medium includes, but is not limited to, any type of disk (including floppy disks, hard disks, optical disks, CD-ROMs, and magneto-optical disks), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic cards, or optical cards. In other words, the storage device includes any medium that stores or transmits information in a readable form by a device (e.g., a computer, a mobile phone), and can be a read-only memory, a disk, or an optical disk, etc.

[0055] Furthermore, the above provides a detailed description of a defect detection method and related apparatus for complex textured surfaces provided by the embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method of defect detection of a complex textured surface, characterized in that, The method comprises: obtaining a surface image of a complex texture product based on an image acquisition module, and pre-processing the surface image to obtain a pre-processed surface image; slicing the pre-processed surface image to obtain a plurality of sub-image blocks; dividing the plurality of sub-image blocks into a plurality of processing image block groups, and distributing each processing image block group to a corresponding artificial intelligence (AI) processor core; each AI processor core uses a defect detection model optimized based on a neural network heterogeneous computing architecture (CANN) to analyze defects in each sub-image block in the processing image block group, and obtains corresponding defect information; fusing the defect information of each sub-image block based on a non-maximum suppression algorithm to obtain global defect information; based on the global defect information, performing defect classification judgment to obtain a defect classification judgment result, and based on the defect classification judgment result, performing product decision execution and model optimization.

2. The method of defect detection of complex textured surfaces according to claim 1, characterized in that, The pre-processing of the surface image to obtain a pre-processed surface image comprises: performing image enhancement processing on the surface image to obtain an image-enhanced surface image; performing normalization processing on the image-enhanced surface image to obtain a normalized surface image; performing size adjustment processing on the normalized surface image to obtain a pre-processed surface image.

3. The method of defect detection of complex textured surfaces according to claim 1, wherein, The slicing of the pre-processed surface image to obtain a plurality of sub-image blocks comprises: obtaining a model target input size and a number of computing nodes of an AI cluster; based on the model target input size and the number of computing nodes, slicing the pre-processed surface image to obtain a plurality of sub-image blocks.

4. The method of defect detection of complex textured surfaces as claimed in claim 1 wherein, The division of the plurality of sub-image blocks into a plurality of processing image block groups and the distribution of each processing image block group to a corresponding AI processor core comprise: determining a plurality of AI processor cores in an idle state, and dividing the plurality of sub-image blocks into a plurality of processing image block groups based on the number of AI processor cores in the idle state; based on a resource scheduling system, distributing each processing image block group to a corresponding AI processor core in a state.

5. The method of defect detection of complex textured surfaces as claimed in claim 1 wherein, Each AI processor core uses a defect detection model optimized based on a neural network heterogeneous computing architecture (CANN) to analyze defects in each sub-image block in the processing image block group, and obtains corresponding defect information, which comprises: collecting a product sample image set and labeling the product sample image set for defect location and defect category to obtain a labeled product sample image set; based on the labeled product sample image set, training a convolutional neural network model using a defect sensitivity loss function, wherein the convolutional neural network comprises a DarkNet53 backbone network, a multi-scale feature fusion neck network, and a multi-classification regression network; based on CANN, compiling and optimizing the trained convolutional neural network model to obtain a defect detection model, and deploying the defect detection model to each corresponding AI processor core; The sub-image blocks in each processing image block group are input into a defect detection model of each AI processor core for defect category, defect position and confidence score analysis, to obtain a detection frame and a confidence score of each sub-image block, and generate defect information based on the detection frame and the confidence score.

6. The method of defect detection of complex textured surfaces as claimed in claim 1 wherein, The defect information of each sub-image block is fused based on a non-maximum suppression algorithm to obtain global defect information, including: An overlapping area between the sub-image blocks is obtained, and a detection frame in each defect information of the overlapping area is determined; The confidence scores of each detection frame in the overlapping area are weighted and fused based on an attention algorithm to obtain a first confidence score of each detection frame; A visibility ratio of each detection frame in the overlapping area is obtained, and a non-maximum suppression threshold is determined based on the first confidence score and the visibility ratio of each detection frame in the overlapping area using a non-maximum suppression threshold analysis model; A target detection frame of the overlapping area is determined based on the first confidence score and the non-maximum suppression threshold; Position mapping and information fusion processing are performed based on the target detection frame of the overlapping area and the defect information of each sub-image block to obtain global defect information.

7. The method of defect detection of complex textured surfaces according to claim 6, wherein, The confidence scores of each detection frame in the overlapping area are weighted and fused based on an attention algorithm to obtain a first confidence score of each detection frame, including: A complete intersection and union ratio loss function of each detection frame in the overlapping area is determined; The attention weight of the confidence score is determined based on the complete intersection and union ratio loss function, and the attention weight and the confidence score are weighted and fused to obtain the first confidence score.

8. The method of defect detection of complex textured surfaces as claimed in claim 1 wherein, The global defect information is used to perform defect classification judgment based on a preset judgment rule to obtain a defect classification judgment result, and product decision execution and model optimization are performed based on the defect classification judgment result, including: The global defect information is used to perform defect classification judgment on a complex texture product based on a preset judgment rule to obtain a defect classification judgment result; A control instruction is generated based on the defect classification judgment result, and the control instruction is sent to a programmable logic controller of a production line, which controls a corresponding device of the production line to execute the control instruction; It is judged whether model optimization is needed based on the defect classification judgment result, and if it is judged that model optimization is needed, the defect classification judgment result is transmitted to an offline training module, and the defect detection model is learned and optimized based on the offline training model using the defect classification judgment result.

9. A device for detecting defects in a complex textured surface, characterized in that, The device includes: An image preprocessing module for obtaining a surface image of a complex texture product based on an image acquisition module and preprocessing the surface image to obtain a preprocessed surface image; An image slicing module for slicing the preprocessed surface image to obtain a plurality of sub-image blocks; An image block distribution module for dividing the plurality of sub-image blocks into a plurality of processing image block groups, and distributing each processing image block group to a corresponding artificial intelligence (AI) processor core. The defect analysis module is configured to perform defect analysis on each sub-image block in the processing image block group by using a defect detection model optimized based on a neural network heterogeneous computing architecture CANN, and obtain corresponding defect information. The information fusion module is configured to perform fusion processing on the defect information of each sub-image block based on a non-maximum suppression algorithm, and obtain global defect information. The defect decision module is configured to perform defect classification judgment based on the global defect information, obtain a defect classification judgment result, and perform product decision execution and model optimization based on the defect classification judgment result.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions, and when the computer instructions run on the electronic device, the electronic device performs the defect detection method of the complex texture surface as claimed in any one of claims 1 to 8.

Images