Non-Lambert surface micro-flaw identification method and system, electronic equipment and storage medium

Abstract

The invention discloses a non-Lambert surface micro-flaw identification method and system, electronic equipment and a storage medium, and the method comprises the steps: inputting a to-be-detected non-Lambert surface image into a preset residual network model for feature extraction, and obtaining a plurality of first feature maps of different scales; performing information fusion on the first feature map through a preset multi-scale information interaction module to obtain a second feature map; performing global information perception fusion on the second feature map through a preset self-attention module to obtain a third feature map; and performing global refinement prediction on the third feature map through a preset multi-view residual full-space information refinement module to obtain a target recognition result. According to the embodiment of the invention, the accuracy of non-Lambert surface micro-flaw recognition can be effectively improved, and the false detection rate and the omission rate of non-Lambert surface micro-flaw recognition are effectively reduced. The method can be widely applied to the technical field of high-end instrument detection.

Description

Technical Field

[0001] This application relates to the field of high-end instrument testing technology, and in particular to a method, system, electronic device and storage medium for identifying micro-defects on non-Lambertian surfaces. Background Art

[0002] Micro-defect identification on non-Lambertian surfaces is an important research direction in industrial vision inspection. Its aim is to accurately identify defects on surfaces with non-uniform reflective properties under complex lighting conditions. In industrial production, micro-defect identification on non-Lambertian surfaces is crucial for product quality control, and achieving this goal requires overcoming the challenges posed by the surface's reflective characteristics. In related technologies, micro-defect identification on non-Lambertian surfaces is easily affected by changes in lighting and surface texture. Furthermore, micro-defects are typically small in size and have low contrast with the background, making it difficult for traditional detection methods to accurately identify these subtle defects, leading to high false positive and false negative rates.

[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention

[0004] The main objective of this application is to provide a method, system, electronic device, and storage medium for identifying micro-defects on non-Lambertian surfaces, which can effectively improve the accuracy of identifying micro-defects on non-Lambertian surfaces and effectively reduce the false detection rate and false negative rate of identifying micro-defects on non-Lambertian surfaces.

[0005] To achieve the above objectives, one aspect of this application proposes a method for identifying micro-defects on non-Lambertian surfaces, the method comprising the following steps: The non-Lambertian surface image to be detected is input into a preset residual network model for feature extraction, resulting in several first feature maps at different scales. The first feature map is fused using a pre-set multi-scale information interaction module to obtain the second feature map; A third feature map is obtained by performing global information perception fusion on the second feature map using a preset self-attention module; The target recognition result is obtained by globally refining and predicting the third feature map through a preset multi-view residual full-space information refinement module.

[0006] In some embodiments, the step of inputting the non-Lambertian surface image to be detected into a preset residual network model for feature extraction to obtain several first feature maps of different scales includes: Construct a ResNet50 model based on preset resolution hierarchical parameters; The non-Lambertian surface image to be detected is input into the ResNet50 model for multi-scale feature extraction to obtain the first feature map.

[0007] In some embodiments, the step of fusing information from the first feature map through a preset multi-scale information interaction module to obtain a second feature map includes: The preset multi-scale information interaction module is constructed based on the Transformer model architecture; wherein, the preset multi-scale information interaction module includes a linear interpolation submodule and several query key-value parameter clusters; The first feature map is spatially scale aligned using the linear interpolation submodule to obtain the fourth feature map. Channel mapping is performed on the fourth feature map to obtain a first mapping result; wherein, the first mapping result includes the query vector, key vector and value vector corresponding to each query key parameter cluster; The query vector corresponding to the query key value parameter cluster is used to calculate the attention score with the key vector corresponding to the other query key value parameter clusters. Then, matrix multiplication is performed on the calculated attention score and the value vector corresponding to the query key value parameter cluster to obtain the fifth feature map. The fifth feature map is residually concatenated with the first feature map to obtain the second feature map.

[0008] In some embodiments, the step of performing global information perception fusion on the second feature map through a preset self-attention module to obtain a third feature map includes: Construct a preset self-attention module; wherein, the preset self-attention module includes a channel direction branch, a spatial direction branch, and a global direction branch; The second feature map is then subjected to attention mapping through the channel direction branch to obtain the second mapping result; The second feature map is subjected to attention mapping through the spatial direction branch to obtain a third mapping result; wherein, the spatial direction branch includes a second dilated convolutional layer and a second activation layer; The second feature map is then subjected to attention mapping through the global directional branch to obtain the fourth mapping result; The third feature map is obtained by fusing information based on the second mapping result, the third mapping result, and the fourth mapping result.

[0009] In some embodiments, constructing the preset self-attention module includes: The channel direction branch is constructed based on the first convolutional layer, the first normalization layer, the first activation layer, the first dilated convolutional layer, and the first pooling layer; The spatial direction branch is constructed based on the second dilated convolutional layer, the second normalization layer, and the second activation layer; The global directional branch is constructed based on the second convolutional layer, the third normalization layer, the third activation layer, and the first fully connected layer; The preset self-attention module is constructed based on the channel direction branch, the spatial direction branch, and the global direction branch.

[0010] In some embodiments, the step of performing global refinement prediction on the third feature map using a preset multi-view residual full-space information refinement module to obtain the target recognition result includes: Construct the preset multi-view residual full-space information refinement module; wherein, the preset multi-view residual full-space information refinement module includes a global information feature processing submodule and a preset channel convolution; The target feature map is obtained by refining the feature information of the third feature map through the global information feature processing submodule. By performing defect discrimination analysis on the target feature map through pre-defined channel convolution, the target recognition result is obtained.

[0011] In some embodiments, constructing the preset multi-view residual full-space information refinement module includes: The first processing branch is constructed based on the third convolutional layer, the fourth normalization layer, and the fourth activation layer; The second processing branch is constructed based on the second pooling layer, the second fully connected layer, the fifth activation layer, the fifth normalization layer, and the linear interpolation layer; The third processing branch is constructed based on the third dilated convolutional layer, the sixth normalization layer, and the sixth activation layer. The outputs and inputs of the first processing branch, the second processing branch, and the third processing branch are residually connected to construct the global information feature processing submodule. The preset multi-view residual full-space information refinement module is constructed based on the global information feature processing submodule and the preset channel convolution.

[0012] To achieve the above objectives, another aspect of this application proposes a non-Lambertian surface micro-defect identification system, the system comprising: The first module is used to input the non-Lambertian surface image to be detected into a preset residual network model for feature extraction, and obtain several first feature maps at different scales. The second module is used to fuse the information of the first feature map through a preset multi-scale information interaction module to obtain a second feature map. The third module is used to perform global information perception fusion on the second feature map through a preset self-attention module to obtain the third feature map; The fourth module is used to perform global refinement prediction on the third feature map through a preset multi-view residual full-space information refinement module to obtain the target recognition result.

[0013] To achieve the above objectives, another aspect of this application provides an electronic device, the electronic device comprising: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor performs the method described above.

[0014] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0015] The embodiments of this application include at least the following beneficial effects: This application provides a method, system, electronic device, and storage medium for identifying micro-defects on non-Lambertian surfaces. This scheme extracts features from an image of a non-Lambertian surface to be detected using a preset residual network model, obtaining several first feature maps at different scales. Then, a preset multi-scale information interaction module fuses the first feature maps to obtain a second feature map. Next, the embodiments of this invention use a preset self-attention module to perform global information perception fusion on the second feature map to obtain a third feature map. Finally, a preset multi-view residual full-space information refinement module performs global refinement prediction on the third feature map to obtain the target recognition result, thus achieving the identification of micro-defects on non-Lambertian surfaces. It is readily understood that, by combining a preset multi-scale information interaction module, a preset self-attention module, and a preset multi-view residual full-space information refinement module for feature processing, the embodiments of the present invention can achieve contrast enhancement of low-contrast sub-regions under high light pollution areas, and can fuse global information to identify high light pollution areas, thereby achieving full-level and full-grained perception of the non-Lambertian surface image to be detected. This can effectively improve the accuracy of non-Lambertian surface micro-defect identification and effectively reduce the false detection rate and false negative rate of non-Lambertian surface micro-defect identification. Attached Figure Description

[0016] Figure 1 This is a flowchart of the non-Lambertian surface micro-defect identification method provided in the embodiments of the present invention; Figure 2 This is a schematic diagram of the overall structure for identifying non-Lambertian surface micro-defects provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the preset multi-scale information interaction module provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the learnable parameter cluster of the preset multi-scale information interaction module provided in the embodiments of the present invention; Figure 5 This is a schematic diagram of a preset self-attention module provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the preset multi-view residual full-space information refinement module provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of the non-Lambertian surface micro-defect identification system provided in an embodiment of the present invention; Figure 8 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0018] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0019] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0021] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained first. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0022] Non-Lambertian surfaces are surfaces that do not follow Lambert's cosine law. Their reflection or radiation characteristics are direction-dependent, such as specular reflection, anisotropic reflection, and retroreflection.

[0023] Residual Network (ResNet) is a deep convolutional neural network architecture. Its core innovation is to solve the gradient vanishing / exploding problem in the training of deep neural networks by introducing a residual learning mechanism.

[0024] The Transformer model is a neural network architecture based on the self-attention mechanism. Its core innovation is that it completely abandons the sequential computation or local connection limitations of recurrent neural networks (RNNs) and convolutional neural networks (CNNs), and achieves efficient parallel sequence modeling.

[0025] Micro-defect identification on non-Lambertian surfaces is an important research direction in industrial vision inspection. Its aim is to accurately identify defects on surfaces with non-uniform reflective properties under complex lighting conditions. In industrial production, micro-defect identification on non-Lambertian surfaces is crucial for product quality control, and achieving this goal requires overcoming the challenges posed by the surface's reflective characteristics. In related technologies, micro-defect identification on non-Lambertian surfaces is easily affected by changes in lighting and surface texture. Furthermore, micro-defects are typically small in size and have low contrast with the background, making it difficult for traditional detection methods to accurately identify these subtle defects, leading to high false positive and false negative rates.

[0026] In view of this, this application provides a method, system, electronic device, and storage medium for identifying micro-defects on non-Lambertian surfaces. This scheme extracts features from the image of the non-Lambertian surface to be detected using a preset residual network model, obtaining several first feature maps at different scales. Then, a preset multi-scale information interaction module fuses the first feature maps to obtain a second feature map. Next, this embodiment of the invention uses a preset self-attention module to perform global information perception fusion on the second feature map to obtain a third feature map. Finally, a preset multi-view residual full-space information refinement module performs global refinement prediction on the third feature map to obtain the target recognition result, thus achieving the identification of micro-defects on non-Lambertian surfaces. This effectively improves the accuracy of non-Lambertian surface micro-defect identification and effectively reduces the false detection rate and false negative rate of non-Lambertian surface micro-defect identification.

[0027] The non-Lambertian surface micro-defect identification method provided in this application relates to the field of high-end instrument testing technology. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the non-Lambertian surface micro-defect identification method, but is not limited to the above forms.

[0028] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0029] Figure 1 This is an optional flowchart of the non-Lambertian surface micro-defect identification method provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps S110 to S140.

[0030] Step S110: Input the non-Lambertian surface image to be detected into the preset residual network model for feature extraction to obtain several first feature maps of different scales.

[0031] Step S120: The first feature map is fused using a preset multi-scale information interaction module to obtain the second feature map.

[0032] Step S130: The second feature map is subjected to global information perception fusion through a preset self-attention module to obtain the third feature map.

[0033] Step S140: The third feature map is globally refined and predicted by the preset multi-view residual full-space information refinement module to obtain the target recognition result.

[0034] In this specific embodiment, the non-Lambertian surface image to be detected refers to the image for which non-Lambertian surface micro-defect detection is required. The preset residual network model refers to the network model constructed by pre-training the residual network model. Specifically, this embodiment extracts multi-scale feature information from the non-Lambertian surface image to be detected using the preset residual network model, obtaining several first feature maps at different scales. For example, this embodiment generates multi-scale feature maps, i.e., several first feature maps at different scales, by performing staged downsampling on the non-Lambertian surface image to be detected using the preset residual network model. Furthermore, in this embodiment, the preset multi-scale information interaction module is an attention mechanism architecture using an end-to-end forward mapping architecture. Specifically, this embodiment inputs the extracted first feature maps at different scales into the preset multi-scale information interaction module to fuse global information, further extracting complex features and patterns in the image, achieving efficient information processing and feature extraction. Then, in this embodiment, the preset self-attention module is a self-attention module with shared weights. Specifically, after passing through the preset multi-scale information interaction module, this embodiment of the invention inputs the obtained multi-scale information into a self-attention module with shared weights, namely, a self-attention module for multi-attention fusion of full-scene information (preset self-attention module), to perform multi-attention aggregation on the input feature information, that is, to perform multi-level attention aggregation, thereby obtaining the third feature map. Finally, this embodiment of the invention performs global refinement prediction on the third feature map through a preset multi-view residual full-space information refinement module to obtain the target recognition result. Specifically, after performing global information perception fusion on the second feature map through the preset self-attention module, this embodiment of the invention adjusts the output information through global refinement prediction, that is, it comprehensively refines and optimizes the features of the third feature map through the preset multi-view residual full-space information refinement module, and then performs micro-defect identification and judgment based on the optimization result to obtain the target recognition result.

[0035] In some embodiments of the present invention, the image of the non-Lambertian surface to be detected is input into a preset residual network model for feature extraction to obtain several first feature maps of different scales, including but not limited to the following steps: A ResNet50 model is constructed based on preset resolution hierarchy parameters.

[0036] The non-Lambertian surface image to be detected is input into the ResNet50 model for multi-scale feature extraction to obtain the first feature map.

[0037] In this specific embodiment, the present invention first constructs a ResNet50 model based on preset resolution hierarchy parameters, and then inputs the non-Lambertian surface image to be detected into the constructed ResNet50 model to perform multi-scale feature extraction to obtain a first feature map. Specifically, in this embodiment, the preset resolution hierarchy parameters refer to parameters used to adjust the scale of the feature map output by the model, such as predefined downsampling positions or frequencies. For example, as shown... Figure 2 As shown, in this embodiment of the invention, the input information is non-Lambertian surface image data collected in an industrial setting. The data is affected by high light pollution, resulting in low contrast in areas with minor defects. This embodiment constructs a ResNet50 model as a feature extractor based on preset resolution hierarchy parameters. The constructed ResNet50 model extracts multi-scale feature information from the input image, including feature maps at 1 / 8, 1 / 16, and 1 / 32 resolutions. Through these feature maps, this embodiment can capture image information at different scales to support subsequent processing steps.

[0038] In some embodiments of the present invention, a second feature map is obtained by fusing information from a first feature map through a preset multi-scale information interaction module, including but not limited to the following steps: A pre-defined multi-scale information interaction module is constructed based on the Transformer model architecture. This module includes a linear interpolation submodule and several query key-value parameter clusters.

[0039] The first feature map is spatially scale aligned using a linear interpolation submodule to obtain the fourth feature map.

[0040] Channel mapping is performed on the fourth feature map to obtain the first mapping result. The first mapping result includes the query vector, key vector, and value vector corresponding to each query key-value parameter cluster.

[0041] The attention score is calculated by performing an attention score calculation on the query vector corresponding to the query key value parameter cluster and the key vector corresponding to the other query key value parameter clusters. Then, matrix multiplication is performed on the calculated attention score and the value vector corresponding to the query key value parameter cluster to obtain the fifth feature map.

[0042] The second feature map is obtained by performing a residual concatenation between the fifth feature map and the first feature map.

[0043] In this specific embodiment, the present invention constructs a preset multi-scale information interaction module based on the Transformer model architecture to fuse the extracted multi-scale feature maps. Specifically, to enhance the expressive power of the feature maps, the present invention integrates global context information and processes the feature maps through a self-attention mechanism. The self-attention mechanism dynamically adjusts the weights of each position in the feature map by calculating the correlation between them, thereby effectively identifying and extracting key high-light pollution areas. This process improves the detection accuracy of micro-defects under complex lighting conditions and ensures reliable defect identification under non-Lambertian surface conditions. Accordingly, the preset multi-scale information interaction module constructed in this embodiment includes a linear interpolation submodule and several query key-value parameter clusters. Firstly, the present invention performs spatial scale alignment processing on the first feature map through the linear interpolation module, that is, unifies information at different scales to the same dimension using linear interpolation to obtain a fourth feature map. This step ensures that information at each scale can effectively interact and fuse within the same dimensional space during subsequent processing. Next, this embodiment of the invention performs channel mapping on the obtained fourth feature map to obtain a first mapping result including the query vector, key vector, and value vector corresponding to each query key-value parameter cluster. Specifically, in this embodiment of the invention, the preset multi-scale information interaction module uses a learnable QKV (Query, Key, Value) parameter cluster to implement channel mapping. In this way, the module can dynamically adjust the information weights of each channel, thereby extracting and integrating valuable feature information more accurately.

[0044] Further, in this embodiment of the invention, attention scores are calculated based on the query vector, key vector, and value vector corresponding to each query key-value parameter cluster, and a fifth feature map is generated. Specifically, this embodiment of the invention calculates attention scores by performing matrix multiplication between the query vector corresponding to a certain query key-value parameter cluster and the key vectors corresponding to other query key-value parameter clusters, and then performs matrix multiplication between the calculated attention scores and the value vectors corresponding to that query key-value parameter cluster to obtain the fifth feature map. For example, as shown... Figure 3 As shown, the preset multi-scale information interaction module includes three query key-value parameter clusters. The attention score of each query key-value cluster is calculated using its corresponding query vector and the key vectors of the other two query key-value clusters. Accordingly, the structure of the query key-value clusters in this embodiment is as follows: Figure 4 As shown. Finally, in this embodiment of the invention, the fifth feature map and the first feature map are residually concatenated to obtain the second feature map. Specifically, as shown... Figure 3As shown, in order to maintain the learnability and propagation of gradients, this embodiment of the invention introduces residual connections during the information fusion process. By fusing the input information (first feature map) and the processed information (fifth feature map) in a residual manner, the model can not only better retain the original information, but also effectively avoid the gradient vanishing problem, thereby improving the training efficiency and stability of the model.

[0045] It is readily understood that, unlike traditional matrix-based parameter training methods, the pre-defined multi-scale information interaction module based on the Transformer architecture in this embodiment of the invention employs an innovative strategy that combines the advantages of convolutional networks and fully connected networks to enhance low-resolution information under highly light-polluted environments. Specifically, this module uses convolutional networks to capture local features and spatial information, which is particularly important for processing details and textures in images. Simultaneously, fully connected networks are used to integrate global information, providing a broader contextual understanding. Accordingly, by fusing the learning capabilities of these two network structures, this embodiment of the invention can effectively enhance the quality of low-resolution information under complex conditions of high light pollution. Furthermore, to further improve the resolution and accuracy of the information, this embodiment of the invention also employs a multi-residual aggregation strategy. By introducing multiple residual connections into the network, the module can aggregate information at different levels, thereby achieving high-resolution correction of the information. This multi-level residual aggregation not only improves the model's sensitivity to details but also enhances its adaptability to complex scenes, ensuring high quality and high accuracy of the output information.

[0046] In some embodiments of the present invention, a third feature map is obtained by performing global information perception fusion on the second feature map through a preset self-attention module, including but not limited to the following steps: Construct a pre-defined self-attention module. This module includes channel direction branches, spatial direction branches, and global direction branches.

[0047] The second feature map is then subjected to attention mapping via the channel direction branch to obtain the second mapping result.

[0048] The third mapping result is obtained by performing attention mapping on the second feature map through a spatial directional branch. The spatial directional branch includes a second dilated convolutional layer and a second activation layer.

[0049] The fourth mapping result is obtained by performing attention mapping on the second feature map through a global directional branch.

[0050] The third feature map is obtained by fusing information based on the second, third, and fourth mapping results.

[0051] In this specific embodiment, the present invention first constructs a preset self-attention module including a channel direction branch, a spatial direction branch, and a global direction branch. Then, attention mapping is performed on the second feature map through the channel direction branch, spatial direction branch, and global direction branch respectively to obtain the corresponding mapping results. Information fusion is then performed on the mapping results to obtain the third feature map. Specifically, after passing through a preset multi-scale information interaction module based on the Transformer architecture, the present invention inputs multi-scale information into a preset self-attention module with shared weights, which is a self-attention module for multi-attention fusion of all-scene information. This module performs multi-attention aggregation on the input information, providing attention through three main branches (channel direction branch, spatial direction branch, and global direction branch) and introducing attention weight factors into the residual branches. In this embodiment, the preset self-attention module is a self-attention module for multi-attention fusion of all-scene information, and its core purpose is to perform multi-level attention aggregation on the input information. Correspondingly, this self-attention module ensures the consistency and coordination of information at different scales during processing through a shared weight mechanism. The internal structure of the module enables in-depth analysis and comprehensive processing of the input information. In this way, the pre-defined self-attention module can identify key features and patterns in information, thereby optimizing and enhancing information globally. To achieve effective aggregation of multiple attention mechanisms, the module constructs three main branches, each responsible for providing attention from different dimensions and angles. These branches capture different features and details in the information through independent attention mechanisms. Then, these attention weight factors are introduced into the residual branch, forming a comprehensive attention enhancement mechanism.

[0052] In some embodiments of the present invention, a preset self-attention module is constructed, including but not limited to the following steps: Channel direction branches are constructed based on the first convolutional layer, the first normalization layer, the first activation layer, the first dilated convolutional layer, and the first pooling layer.

[0053] Spatial directional branches are constructed based on the second dilated convolutional layer, the second normalized layer, and the second activation layer.

[0054] A global directional branch is constructed based on the second convolutional layer, the third normalization layer, the third activation layer, and the first fully connected layer.

[0055] A preset self-attention module is constructed based on the channel direction branch, spatial direction branch, and global direction branch.

[0056] In this specific embodiment, the channel direction branch is constructed from a first convolutional layer, a first normalization layer, a first activation layer, a first dilated convolutional layer, and a first pooling layer. Specifically, as shown... Figure 5As shown, in this embodiment of the invention, the channel-direction branch uses 3×3 convolution, normalization layer, activation layer, dilated convolution, pooling convolution, etc., to perform attention space mapping in the channel direction. Specifically, in the attention space mapping in the channel direction, this embodiment constructs a combination of 3×3 convolution, normalization layer, and activation layer. This combination effectively captures the correlation between channels, and the normalization layer standardizes the data, ensuring the stability and consistency of information during transmission. In this embodiment, the activation layer enhances the model's ability to express complex features by introducing nonlinear transformations. Furthermore, the application of dilated convolution and pooling convolution further expands the receptive field, enabling the model to capture a wider range of contextual information. Further, this embodiment constructs the spatial direction branch through a second dilated convolution layer, a second normalization layer, and a second activation layer. Specifically, as... Figure 5 As shown, this embodiment of the invention achieves spatial attention mapping through multiple dilated convolutions and activation layers. The use of multiple dilated convolutions and activation layers is crucial in spatial attention mapping. Specifically, in this embodiment, the dilated convolution expands the receptive field of the convolution by introducing holes in the convolution kernel without increasing computational complexity. This allows the model to capture spatial features over a wider range, enhancing its sensitivity to image details. Furthermore, the activation layer continues to provide non-linear transformations, ensuring the diversity and richness of spatial features. Accordingly, in this embodiment, the global directional branch is constructed through a second convolutional layer, a third normalization layer, a third activation layer, and a first fully connected layer. Specifically, as... Figure 5 As shown, this embodiment of the invention employs fully connected layers to achieve global information mapping. The introduction of fully connected layers plays a crucial role in this global information mapping. Specifically, by connecting all input nodes to output nodes, the fully connected layer achieves global integration and mapping of information. This global perspective allows the model to understand and optimize the input information holistically, ensuring the comprehensiveness and accuracy of the output results.

[0057] In some embodiments of the present invention, a target recognition result is obtained by globally refining the third feature map through a preset multi-view residual full-space information refinement module, including but not limited to the following steps: A pre-defined multi-view residual full-space information refinement module is constructed. This module includes a global information feature processing sub-module and a pre-defined channel convolution.

[0058] The target feature map is obtained by refining the feature information of the third feature map through the global information feature processing submodule.

[0059] By performing defect discrimination analysis on the target feature map through convolution with preset channels, the target recognition result is obtained.

[0060] In this specific embodiment, the present invention first constructs a preset multi-view residual spatial information refinement module, including a global information feature processing submodule and a preset channel convolution. Then, the global information feature processing submodule refines the feature information of the third feature map, and the preset channel convolution performs defect discrimination analysis on the processed target feature map to obtain the target recognition result. Specifically, after the self-attention module of full-scene information multi-attention fusion achieves global information perception fusion of the feature map, the present invention uses a global refinement prediction method to make a final adjustment to the output information, namely, the preset multi-view residual full-space information refinement module. The global refinement prediction method in this embodiment aims to ensure the accuracy and detail fidelity of the output result by performing more refined processing on the fused feature information. This method utilizes the concept of multi-view, that is, re-examining and adjusting information at different scales and perspectives. The global information feature processing submodule refines the feature information, and this multi-view processing approach enables the model to capture subtle differences and potential patterns in the information from different angles, thereby improving the accuracy of prediction. Finally, in this embodiment of the invention, a defect discrimination analysis is performed on the processed target feature map by using a preset channel convolution, such as a 1×1 channel convolution, to achieve the discrimination and classification of micro-defect categories.

[0061] In some embodiments of the present invention, a preset multi-view residual full-space information refinement module is constructed, including but not limited to the following steps: The first processing branch is constructed based on the third convolutional layer, the fourth normalization layer, and the fourth activation layer.

[0062] The second processing branch is constructed based on the second pooling layer, the second fully connected layer, the fifth activation layer, the fifth normalization layer, and the linear interpolation layer.

[0063] The third processing branch is constructed based on the third dilated convolutional layer, the sixth normalization layer, and the sixth activation layer.

[0064] By performing residual connections between the outputs and inputs of the first, second, and third processing branches, a global information feature processing submodule is constructed.

[0065] Based on the global information feature processing submodule and the preset channel convolution, a preset multi-view residual full-space information refinement module is constructed.

[0066] In this specific embodiment, the first processing branch of the present invention is constructed through a third convolutional layer, a fourth normalization layer, and a fourth activation layer. For example... Figure 6As shown, in this embodiment of the invention, the third convolutional layer uses a 3×3 convolution. In the first processing branch, the 3×3 convolution, normalization layer, and activation layer are connected sequentially. Furthermore, in this embodiment of the invention, the second processing branch is constructed using a second pooling layer, a second fully connected layer, a fifth activation layer, a fifth normalization layer, and a linear interpolation layer. For example, as... Figure 6 As shown, in the second processing branch, the global pooling layer, fully connected layer, activation layer, fully connected layer, normalization layer, and linear interpolation layer are connected sequentially. Meanwhile, in this embodiment of the invention, the third processing branch is constructed through a third dilated convolutional layer, a sixth normalization layer, and a sixth activation layer. For example, as... Figure 6 As shown, in the third processing branch, the dilated convolutional layer, normalization layer, activation layer, dilated convolutional layer, normalization layer, and activation layer are connected sequentially. Further, in this embodiment of the invention, the outputs and inputs of the first, second, and third processing branches are residually connected; that is, the input information of the preset multi-view residual full-space information module is residually connected with the output information of the three branches, as shown below. Figure 6 As shown, this embodiment of the invention provides strong support for information refinement through the introduction of residual structures. By introducing skip connections, the residual structure allows information to retain its original features during transmission, while also allowing the model to adjust and optimize features at a higher level. This structure not only improves the training efficiency of the model but also effectively prevents problems such as gradient vanishing. Correspondingly, the pre-set multi-view residual full-space information refinement module in this embodiment of the invention ensures the comprehensiveness and consistency of the output results through comprehensive analysis and processing of information. This module comprehensively considers the interrelationships between spatial, channel, and global features, and uses complex calculation and optimization strategies to make meticulous adjustments to the information. This module uses residual connection methods to effectively fuse input features with processed features. In this embodiment of the invention, the preset field-of-view residual full-space information refinement module employs a residual connection method. It uses cascaded 3×3 convolutions to focus on small fields of view in the surrounding area, dilated convolutions to recognize multiple surrounding fields of view, and global pooling and fully connected layers to perceive global information. Finally, it combines preset channel convolutions to construct the preset multi-field-of-view residual full-space information refinement module, enabling the discrimination and classification of micro-defect categories. This residual connection not only facilitates the smooth transmission of information but also preserves original feature information, avoiding information loss in deep networks.

[0067] It is readily understood that this embodiment of the invention employs cascaded 3×3 convolutions to focus on the small field of view in the surrounding area. This convolution operation, by extracting features within local regions, effectively captures subtle spatial changes and local patterns, thereby enhancing the model's sensitivity to details and its recognition capabilities. Simultaneously, to expand the model's perceptual range, the module also introduces dilated convolutions. This convolution technique introduces holes between convolution kernels, allowing the model to expand its receptive field without increasing the number of parameters, thus achieving recognition of multiple surrounding visual regions. The application of dilated convolutions enables the model to capture contextual information over a larger scope, enhancing its understanding of complex scenes. Furthermore, the combination of global pooling and fully connected layers provides the model with the ability to perceive global information. Global pooling summarizes the entire feature map to extract global features, while fully connected layers further process and analyze these features to achieve a comprehensive understanding of the overall information. This process ensures that the model can balance global and local information during refinement. Finally, the module uses 1×1 channel convolutions to discriminate and classify micro-defect categories. Correspondingly, 1×1 convolution, through linear transformation along the channel dimension, can effectively integrate information from different channels, thereby achieving accurate identification and classification of specific categories. This operation not only improves the model's classification accuracy but also enhances its ability to detect subtle defects.

[0068] For example, the non-Lambertian surface micro-defect recognition method provided in this embodiment of the invention is compared with multiple methods on two datasets (mIou) as shown in Table 1. Experiments show that the non-Lambertian surface micro-defect recognition method provided in this embodiment of the invention can significantly improve the accuracy of semantic segmentation of metal surface defects.

[0069] Table 1

[0070] It is readily understood that the embodiments of the present invention, through a multi-scale information interaction module based on the Transformer architecture, a self-attention module that fuses multi-attention information across the entire scene, and a multi-view residual full-space information refinement module, achieve contrast enhancement for low-contrast micro-defect areas in areas of high light pollution. Furthermore, it can fuse global information to identify areas of high light pollution, achieving full-level, fine-grained perception of the input non-Lambertian surface image. In summary, the non-Lambertian surface micro-defect recognition method provided by the embodiments of the present invention significantly improves the ability to recognize non-Lambertian surface micro-defects in areas of high light pollution by expanding the receptive field of existing methods and fully utilizing information from the channel direction.

[0071] Please see Figure 7 This application also provides a non-Lambertian surface micro-defect identification system, which can implement the above-described non-Lambertian surface micro-defect identification method. The system includes: The first module 210 is used to input the non-Lambertian surface image to be detected into a preset residual network model for feature extraction, and obtain several first feature maps of different scales. The second module 220 is used to fuse the first feature map with information through a preset multi-scale information interaction module to obtain a second feature map. The third module 230 is used to perform global information perception fusion on the second feature map through a preset self-attention module to obtain the third feature map; The fourth module 240 is used to perform global refinement prediction on the third feature map through a preset multi-view residual full-space information refinement module to obtain the target recognition result.

[0072] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0073] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method for identifying non-Lambertian surface micro-defects. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0074] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0075] Please see Figure 8 , Figure 8 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 310 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 320 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 320 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 320 and called by the processor 310 to execute the non-Lambertian surface micro-defect identification method of the embodiments of this application. Input / output interface 330 is used to realize information input and output; The communication interface 340 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 350 transmits information between various components of the device (e.g., processor 310, memory 320, input / output interface 330, and communication interface 340); The processor 310, memory 320, input / output interface 330 and communication interface 340 are connected to each other within the device via bus 350.

[0076] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for identifying non-Lambertian surface micro-defects.

[0077] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0078] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0079] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0080] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0081] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0082] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0083] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0084] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0085] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0086] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0087] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0088] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0089] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for identifying micro-defects on non-Lambertian surfaces, characterized in that, The method includes the following steps: The non-Lambertian surface image to be detected is input into a preset residual network model for feature extraction, resulting in several first feature maps at different scales. The first feature map is fused using a pre-set multi-scale information interaction module to obtain the second feature map; A third feature map is obtained by performing global information perception fusion on the second feature map using a preset self-attention module; The target recognition result is obtained by globally refining and predicting the third feature map through a preset multi-view residual full-space information refinement module.

2. The method according to claim 1, characterized in that, The process involves inputting the non-Lambertian surface image to be detected into a preset residual network model for feature extraction, resulting in several first feature maps at different scales, including: Construct a ResNet50 model based on preset resolution hierarchical parameters; The non-Lambertian surface image to be detected is input into the ResNet50 model for multi-scale feature extraction to obtain the first feature map.

3. The method according to claim 1, characterized in that, The step of fusing information from the first feature map through a preset multi-scale information interaction module to obtain a second feature map includes: The preset multi-scale information interaction module is constructed based on the Transformer model architecture; wherein, the preset multi-scale information interaction module includes a linear interpolation submodule and several query key-value parameter clusters; The first feature map is spatially scale aligned using the linear interpolation submodule to obtain the fourth feature map. Channel mapping is performed on the fourth feature map to obtain a first mapping result; wherein, the first mapping result includes the query vector, key vector and value vector corresponding to each query key parameter cluster; The query vector corresponding to the query key value parameter cluster is used to calculate the attention score with the key vector corresponding to the other query key value parameter clusters. Then, matrix multiplication is performed on the calculated attention score and the value vector corresponding to the query key value parameter cluster to obtain the fifth feature map. The fifth feature map is residually concatenated with the first feature map to obtain the second feature map.

4. The method according to claim 1, characterized in that, The step of performing global information perception fusion on the second feature map through a preset self-attention module to obtain the third feature map includes: Construct a preset self-attention module; wherein, the preset self-attention module includes a channel direction branch, a spatial direction branch, and a global direction branch; The second feature map is then subjected to attention mapping through the channel direction branch to obtain the second mapping result; The second feature map is subjected to attention mapping through the spatial direction branch to obtain a third mapping result; wherein, the spatial direction branch includes a second dilated convolutional layer and a second activation layer; The second feature map is then subjected to attention mapping through the global directional branch to obtain the fourth mapping result; The third feature map is obtained by fusing information based on the second mapping result, the third mapping result, and the fourth mapping result.

5. The method according to claim 4, characterized in that, The construction of the preset self-attention module includes: The channel direction branch is constructed based on the first convolutional layer, the first normalization layer, the first activation layer, the first dilated convolutional layer, and the first pooling layer; The spatial direction branch is constructed based on the second dilated convolutional layer, the second normalization layer, and the second activation layer; The global directional branch is constructed based on the second convolutional layer, the third normalization layer, the third activation layer, and the first fully connected layer; The preset self-attention module is constructed based on the channel direction branch, the spatial direction branch, and the global direction branch.

6. The method according to claim 1, characterized in that, The step of performing global refinement prediction on the third feature map through a preset multi-view residual full-space information refinement module to obtain the target recognition result includes: Construct the preset multi-view residual full-space information refinement module; wherein, the preset multi-view residual full-space information refinement module includes a global information feature processing submodule and a preset channel convolution; The target feature map is obtained by refining the feature information of the third feature map through the global information feature processing submodule. By performing defect discrimination analysis on the target feature map through pre-defined channel convolution, the target recognition result is obtained.

7. The method according to claim 6, characterized in that, The construction of the preset multi-view residual full-space information refinement module includes: The first processing branch is constructed based on the third convolutional layer, the fourth normalization layer, and the fourth activation layer; The second processing branch is constructed based on the second pooling layer, the second fully connected layer, the fifth activation layer, the fifth normalization layer, and the linear interpolation layer; The third processing branch is constructed based on the third dilated convolutional layer, the sixth normalization layer, and the sixth activation layer. The outputs and inputs of the first processing branch, the second processing branch, and the third processing branch are residually connected to construct the global information feature processing submodule. The preset multi-view residual full-space information refinement module is constructed based on the global information feature processing submodule and the preset channel convolution.

8. A non-Lambertian surface micro-defect identification system, characterized in that, The system includes: The first module is used to input the non-Lambertian surface image to be detected into a preset residual network model for feature extraction, and obtain several first feature maps at different scales. The second module is used to fuse the information of the first feature map through a preset multi-scale information interaction module to obtain a second feature map. The third module is used to perform global information perception fusion on the second feature map through a preset self-attention module to obtain the third feature map; The fourth module is used to perform global refinement prediction on the third feature map through a preset multi-view residual full-space information refinement module to obtain the target recognition result.

9. An electronic device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method as described in any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images