A deep learning-based optical surface defect data detection method

Abstract

This invention discloses a deep learning-based method for detecting optical surface defects, belonging to the field of optical defect detection technology. The method includes: constructing an optical scattering physics model; inputting acquired optical surface images into the optical scattering physics model for multimodal data synthesis to generate multimodal image data; constructing a deep learning feature extraction network; inputting the multimodal image data; and performing multi-scale feature fusion and enhancement through a bidirectional attention feedback mechanism to generate a depth feature map; and using a deep learning region generation method to perform spatial domain analysis on the depth feature map, locating the coordinates of potential defect regions, and generating a set of candidate defect region coordinates. This invention, through multimodal data synthesis driven by an optical scattering physics model, overcomes the limitations of a single imaging mode, dynamically analyzes the scattering characteristics of defects under multi-physics coupling, enhances the identifiability of weak defects in complex scattering environments, and solves the problem of missed defect detection.

Description

Technical Field

[0001] This invention relates to the field of optical defect detection technology, and in particular to a method for detecting optical surface defect data based on deep learning. Background Technology

[0002] In recent years, deep learning technology has made progress in the field of optical surface defect detection. Feature extraction methods based on convolutional neural networks can automatically learn abstract representations of surface textures, overcoming the limitations of traditional hand-designed features. The combination of Region Generation Networks (RPNs) and Feature Pyramid Networks (FPNs) improves the ability to locate defects at multiple scales, while the introduction of attention mechanisms further enhances the ability to focus on subtle defect features. Furthermore, end-to-end detection frameworks improve the robustness of defect identification in complex scattering environments by jointly optimizing localization and classification tasks.

[0003] Existing optical surface defect detection methods have shortcomings. They rely on a single imaging mode and cannot dynamically analyze the multi-physics coupling features of defects. This leads to insufficient signal-to-noise ratio and increased false negative rate for weak defects under specific lighting conditions. They also use a unidirectional feature transfer path and lack the ability to symmetrically interact with features across different levels. High-level semantic features and low-level detail features are fused only through simple skip connections without establishing a bidirectional feedback mechanism. This results in the breakage of contextual associations for multi-scale defects. Unidirectional fusion is difficult to coordinate the consistent expression of local texture and global structure, causing redundancy in localization boxes or fluctuations in classification confidence. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a deep learning-based optical surface defect data detection method to solve the problems of missed detection of weak defects and insufficient cross-scale semantic fusion.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] This invention provides a deep learning-based method for detecting optical surface defects. The method includes: constructing an optical scattering physics model; inputting acquired optical surface images into the optical scattering physics model for multimodal data synthesis to generate multimodal image data; constructing a deep learning feature extraction network and inputting the multimodal image data, performing multi-scale feature fusion and enhancement through a bidirectional attention feedback mechanism to generate a depth feature map; performing spatial domain analysis on the depth feature map using a deep learning region generation method to locate the coordinates of potential defect regions and generate a set of candidate defect region coordinates; cropping local feature maps corresponding to each candidate defect region from the depth feature map and performing scale normalization on the local feature maps to generate a set of candidate region feature maps; compressing each feature map in the candidate region feature map set into a feature vector, and extracting semantic information from the feature vectors through nonlinear transformation and feature dimensionality reduction to generate a set of candidate region feature vectors; inputting the set of candidate region feature vectors into a deep learning classifier and outputting a defect category determination result and a confidence score.

[0008] As a preferred embodiment of the deep learning-based optical surface defect data detection method of the present invention, the specific steps for constructing the optical scattering physical model are as follows:

[0009] A theoretical framework layer is constructed based on the bidirectional reflection distribution function theory, a physical property calculation layer is constructed based on the light scattering characteristics of microsurfaces, and a parameter configuration layer is constructed based on the physical property parameter types.

[0010] An optical scattering physics model is constructed by combining the theoretical framework layer, the physical property calculation layer, and the parameter configuration layer.

[0011] As a preferred embodiment of the deep learning-based optical surface defect data detection method of the present invention, the specific steps for generating multimodal image data are as follows:

[0012] Optical surface images are acquired using an image sensor and a controllable lighting device, and the optical surface images are analyzed using an inverse rendering method to extract physical property parameter values.

[0013] The physical property parameter values ​​are assigned to the parameter configuration layer, the values ​​of the parameter configuration layer are called according to the rules of the theoretical framework layer, Monte Carlo rendering is performed in the physical property calculation layer to generate a composite image under the current lighting conditions; after traversing all preset lighting conditions, multimodal image data is generated.

[0014] As a preferred embodiment of the deep learning-based optical surface defect data detection method of the present invention, the specific steps for constructing the deep learning feature extraction network are as follows:

[0015] A topology layer is constructed based on an encoder-decoder architecture, a feature transformation layer is constructed based on feature extraction and resolution transformation operations, and a feature fusion layer is constructed based on a bidirectional attention feedback mechanism.

[0016] A deep learning feature extraction network is constructed based on a topology layer, a feature transformation layer, and a feature fusion layer.

[0017] As a preferred embodiment of the deep learning-based optical surface defect data detection method of the present invention, the specific steps for generating the depth feature map are as follows:

[0018] Multimodal image data is input into a deep learning feature extraction network, and the multimodal image data is encoded and decoded through a topological structure layer;

[0019] The feature transformation layer performs feature extraction and resolution transformation, while the feature fusion layer generates a depth feature map through feature enhancement and fusion.

[0020] As a preferred embodiment of the deep learning-based optical surface defect data detection method of the present invention, the specific steps for generating the candidate defect region coordinate set are as follows:

[0021] A multi-scale feature pyramid is constructed based on the deep feature map. An anchor box with different size and aspect ratio is preset at each spatial location of the multi-scale feature pyramid to generate an initial set of anchor boxes.

[0022] Based on the multi-scale feature pyramid, the defect confidence score and bounding box coordinate offset of each initial anchor box are calculated to generate a preliminary set of candidate anchor boxes;

[0023] The preliminary candidate anchor boxes are sorted according to the defect confidence score, the selected boxes are retained, and the cross-union ratio is calculated based on the retained boxes. The preliminary candidate anchor boxes whose cross-union ratio does not exceed the preset cross-union ratio threshold are extracted to generate a refined candidate anchor box set.

[0024] Calculate the scaling ratio of the depth feature map relative to the optical surface image, and based on the scaling ratio, forward map the refined candidate anchor box set to the coordinate space of the optical surface image to generate a set of candidate defect region coordinates.

[0025] As a preferred embodiment of the deep learning-based optical surface defect data detection method of the present invention, the specific steps for cropping the local feature map corresponding to each candidate defect region from the depth feature map based on the candidate defect region coordinate set and the depth feature map are as follows:

[0026] Based on the size scaling ratio, each anchor box in the candidate defect region coordinate set is inversely mapped to the depth feature map to generate a depth feature map anchor box set.

[0027] Bilinear interpolation feature clipping is performed on each set of anchor points in the depth feature map to extract the local feature map of the corresponding region.

[0028] As a preferred embodiment of the deep learning-based optical surface defect data detection method of the present invention, the specific steps for scaling the local feature map to generate a candidate region feature map set are as follows:

[0029] Local feature maps of different sizes are normalized to a fixed size using bilinear interpolation;

[0030] All fixed-size local feature maps are integrated in the same order as the candidate defect region coordinate set to generate a candidate region feature map set.

[0031] As a preferred embodiment of the deep learning-based optical surface defect data detection method of the present invention, the specific steps for the candidate region feature vector set are as follows:

[0032] The initial feature vector set is generated by compressing each feature map in the candidate region feature map set into an initial feature vector through a global average pooling operation.

[0033] Each initial feature vector in the initial feature vector set is projected into a high-dimensional space and the ReLU activation function is applied to generate a high-dimensional feature vector.

[0034] Projecting high-dimensional feature vectors onto a low-dimensional space generates dimensionality-reduced feature vectors.

[0035] Batch normalization is performed on the dimensionality-reduced feature vectors to generate standardized feature vectors;

[0036] The standardized feature vectors are sorted according to the original order of the candidate region feature map set to generate a candidate region feature vector set.

[0037] As a preferred embodiment of the deep learning-based optical surface defect data detection method of the present invention, the specific steps of inputting the candidate region feature vector set into the deep learning classifier and outputting the defect category determination result and confidence score are as follows:

[0038] Each candidate region feature vector in the candidate region feature vector set is input into the fully connected layer for calculation to generate an initial score vector for each defect category;

[0039] The initial score vector is converted into a probability distribution vector using the Softmax function, generating the probability value of each candidate region belonging to each defect category;

[0040] Extract the defect category corresponding to the highest probability value as the defect category determination result, and use the highest probability value as the confidence score.

[0041] The beneficial effects of this invention are as follows: By using a multimodal data synthesis driven by an optical scattering physics model, the limitations of a single imaging mode are overcome, the scattering characteristics of defects under multi-physics coupling are dynamically analyzed, the identifiability of weak defects in complex scattering environments is enhanced, and the problem of missed defect detection is solved; by establishing a symmetrical interaction between the encoder and decoder paths through a bidirectional attention feedback mechanism, and by fusing cross-level features through bidirectional weight modulation, the consistent expression of local fine textures and global structural semantics is coordinated, eliminating the redundancy in the location of irregular composite defects and the fluctuation in classification confidence, thus achieving a substantial breakthrough in detection accuracy and robustness. Attached Figure Description

[0042] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a flowchart of a deep learning-based optical surface defect data detection method.

[0044] Figure 2 A flowchart for generating multimodal image data.

[0045] Figure 3 The flowchart for generating depth feature maps.

[0046] Figure 4 This is a flowchart for outputting the defect category determination results and confidence scores. Detailed Implementation

[0047] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0048] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0049] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0050] Reference Figures 1-4 This is one embodiment of the present invention, which provides a method for detecting optical surface defect data based on deep learning, including the following steps:

[0051] S1. Construct an optical scattering physical model, input the collected optical surface images into the optical scattering physical model for multimodal data synthesis, and generate multimodal image data;

[0052] S1.1. Construct a theoretical framework layer based on the bidirectional reflection distribution function theory, a physical property calculation layer based on the light scattering characteristics of micro-surfaces, and a parameter configuration layer based on the physical property parameter types;

[0053] It should be noted that the mathematical expression of the bidirectional reflectance distribution function is used as the mathematical form for light scattering calculations. The bidirectional reflectance distribution function defines the radiative energy mapping relationship between the incident and emitted light. Based on the principle of energy conservation, the bidirectional reflectance distribution function is required to satisfy integral constraints, and the Helmholtz reciprocity principle is followed to ensure mathematical symmetry. Based on the bidirectional reflectance distribution function, a rendering equation is defined, completing the construction of the theoretical framework. The mathematical expression of the bidirectional reflectance distribution function is:

[0054] ;

[0055] = ;

[0056] in, The radiance produced per unit irradiance refers to the radiance produced at a wavelength of [wavelength missing]. Below, the surface material will change from a unit vector of Light incident on the direction of incidence is reflected into a unit vector of... The efficiency of the emission direction, that is, how the surface material converts the incident light energy (irradiance) into the emitted light brightness (radiance). Indicates the direction of light incidence. , unit vector; Indicates the direction of light emission , unit vector; Indicates wavelength; Indicates the emitted radiance; Indicates incident irradiance; Indicates the incident radiation brightness; This indicates the angle of incidence, which is the angle between the incident light and the surface normal. Indicates the direction of incidence; Indicates the direction of launch;

[0057] The expression for the rendering equation is,

[0058] ;

[0059] in, Indicates the direction of the incident light. and the direction of the emitted light The emitted radiation brightness; This represents the surface roughness parameter; Indicates the basic reflectivity parameter; Represents the distribution function of the normal to the micro-surface; ; Represents the Fresnel reflectivity function;

[0060] The bidirectional reflectance distribution function defines the physical principle of light-surface interaction, quantifies the efficiency of a surface material in converting incident light energy into outgoing radiance at a specific wavelength, reflects the radiative energy mapping relationship of light scattering, and follows the principle of energy conservation and the Helmholtz reciprocity principle to ensure physical correctness.

[0061] The microsurface normal distribution function is used to describe the directional influence of surface micro-geometry on light scattering. A geometric attenuation function is introduced to represent the energy attenuation caused by the occlusion effect between microsurfaces. The Fresnel reflectivity function is integrated to calculate the variation of material reflectivity with the incident angle. The microsurface normal distribution function, geometric attenuation function, and Fresnel reflectivity function are integrated through the mathematical framework of a bidirectional reflectance distribution function to form a light scattering property calculation system. The scattering efficiency coefficient is output, completing the construction of the physical property calculation layer. The mathematical expression for the output scattering efficiency coefficient is as follows:

[0062] ;

[0063] ;

[0064] in, This represents the scattering efficiency coefficient; It represents a half-angle vector, that is, the unit vector along the angle bisector of the incident light direction and the outgoing light direction; This refers to the micro-surface normal at half-angle vectors The statistical distribution density in a direction determines the shape and sharpness of the highlight; This refers to the attenuation of light energy caused by the shading effect (shadowing and occlusion) of micro-surfaces; This refers to the change in the reflectivity of a material surface as the angle of incidence of light changes; Represents the unit vector of the macroscopic surface normal;

[0065] Physical property parameters include surface geometric property parameters (surface roughness parameters), material optical property parameters (refractive index parameters and basic reflectivity parameters), and material classification property parameters (metallicity parameters).

[0066] Surface roughness parameters are defined to describe the degree of irregularity in the geometry of micro-surfaces. Refractive index parameters are defined to represent the optical density characteristics of materials. Metallicity parameters are introduced to distinguish the reflection characteristics of conductors and dielectrics. The material's reflectivity under perpendicular incidence is quantified through basic reflectivity parameters. Standardized data storage interfaces and access protocols are created for surface roughness parameters, refractive index parameters, metallicity parameters, and basic reflectivity parameters to form a parameter configuration system and complete the construction of the parameter configuration layer.

[0067] S1.2 Construct an optical scattering physical model based on the theoretical framework layer, physical property calculation layer, and parameter configuration layer;

[0068] It should be noted that the parameter configuration layer provides physical property parameters such as surface roughness, refractive index, metallicity, and basic reflectivity as input data. The theoretical framework layer provides the mathematical expression of the bidirectional reflection distribution function and the constraints of the energy conservation and reciprocity principle. The physical property calculation layer uses the physical property parameter types of the parameter configuration layer and the mathematical expression of the bidirectional reflection distribution function of the theoretical framework layer to perform the calculation of micro-surface normal distribution, geometric attenuation, and Fresnel reflectivity. The three layers complete the construction of the optical scattering physical model through a tightly coupled dependency relationship.

[0069] The optical scattering physical model itself does not require training. The essence of the optical scattering physical model is a deterministic mathematical model based on the laws of physical optics. All the physical property parameter types required by the parameter configuration layer can be obtained by extracting the acquired optical surface images, rather than by learning trainable weights from data through optimization algorithms such as gradient descent. Therefore, the establishment of the optical scattering physical model is the determination and assignment of physical property parameter types. Once the physical property parameter types are determined, the calculation results of the optical scattering physical model are determined by these physical property parameters and fixed mathematical formulas. There is a clear and derivable physical causal relationship between the output and input of the optical scattering physical model, without the need for fitting and training with a large amount of data.

[0070] S1.3 Acquire optical surface images using an image sensor and a controllable lighting device, and analyze the optical surface images using an inverse rendering method to extract physical property parameter values;

[0071] It should be noted that different lighting conditions are generated by actively adjusting the geometric angle, spectral composition, and polarization state of the light through a controllable lighting device. These conditions include changes in the incident angle of the light (such as bright field lighting with vertical incidence and dark field lighting with oblique incidence), differences in lighting methods (such as ring light and coaxial light), switching of spectral bands (such as monochromatic light lighting of different wavelengths), and adjustment of polarization state (such as linear polarization and circular polarization).

[0072] Optical surface image sequences under different lighting conditions are acquired using an image sensor and a controllable illumination device. Initial estimates are assigned to surface roughness, refractive index, metallicity, and basic reflectivity parameters. The initial estimate for surface roughness (e.g., 0.5) is defined based on the commonly used reference median of the root mean square deviation of height distribution in surface topography statistics. The initial estimate for refractive index (e.g., 1.5) is defined based on the inherent refractive constant of optical glass materials in the visible light band. The initial estimate for metallicity (e.g., 0 or 1, a binary description where 0 represents non-conductor and 1 represents conductor) is defined based on the boundary condition theory of electromagnetic wave propagation at the interface of a medium. The initial estimate for reflectivity of dielectric materials under perpendicular incidence in the visible light band (e.g., 0.4) is also defined.

[0073] For each preset lighting condition and the surface point corresponding to each pixel position of the image sensor, the initial estimates of each physical property parameter and the scattering efficiency coefficient are substituted into the rendering equation. The rendering equation is used to calculate the emitted radiance value of the surface point under the current lighting direction and the observation direction. The calculation result of each pixel position is directly used as the pixel radiance value of the corresponding coordinate position in the simulated image. The calculation results of all pixels under each independent lighting condition constitute a complete simulated image. The simulated images corresponding to all lighting conditions are organized in sequence to form a simulated image sequence. The sum of squares of the differences in emitted radiance values ​​at each pixel position between the simulated image sequence and the acquired optical surface image is calculated as the overall difference measure. The gradient descent algorithm is used to calculate the partial derivative of the overall difference measure with respect to the initial estimates of the physical property parameters, and the initial estimates of the physical property parameters are updated in the opposite direction of the gradient. The process of synthesizing the simulated image sequence, calculating the overall difference measure, and updating the initial estimates of the physical property parameters is repeated until the overall difference measure is lower than the convergence tolerance (based on the definition of the iteration termination criterion in numerical optimization theory, such as 10). -6 This completes the extraction of physical property parameter values.

[0074] S1.4 Assign the physical property parameter values ​​to the parameter configuration layer, call the values ​​of the parameter configuration layer according to the rules of the theoretical framework layer, perform Monte Carlo rendering in the physical property calculation layer, and generate a composite image under the current lighting conditions; after traversing all preset lighting conditions, generate multimodal image data.

[0075] It should be noted that the physical attribute parameter values ​​are assigned to the parameter type storage location corresponding to the parameter configuration layer. Based on the mathematical expression of the bidirectional reflection distribution function and the energy conservation constraint rules defined in the theoretical framework layer, the physical attribute parameter values ​​are called from the parameter configuration layer and input into the physical attribute calculation layer. In the physical attribute calculation layer, a large number of incident light direction samples are randomly generated for each pixel location on the surface. Each incident light direction sample and scattering efficiency coefficient are substituted into the rendering equation to generate an estimated value of the outgoing radiance. After traversing all surface points to complete random sampling and substitution into the rendering equation, all the generated estimated values ​​of outgoing radiance are integrated into a synthetic image corresponding to the current specific lighting conditions. The above assignment, value calling and rendering calculation process is repeated for all lighting conditions to generate multimodal image data containing all synthetic images under different lighting conditions.

[0076] It should also be noted that existing technologies encrypt optical surface images using fixed algorithms, which can ensure data security but lack the ability to dynamically analyze the optical properties of materials and cannot adapt to the complex scattering characteristics of optical surface defects. This solution, by constructing an optical scattering physical model, achieves accurate modeling of the physical properties of optical surface defects and multimodal data synthesis, thus solving the problem that encryption methods cannot capture the dynamic scattering characteristics of optical surfaces.

[0077] S2. Construct a deep learning feature extraction network and input multimodal image data. Use a bidirectional attention feedback mechanism to fuse and enhance multi-scale features to generate a deep feature map.

[0078] S2.1. Construct a topology layer based on the encoder-decoder architecture, a feature transformation layer based on feature extraction and resolution transformation operations, and a feature fusion layer based on a bidirectional attention feedback mechanism;

[0079] It should be noted that the encoder is defined as a path consisting of multiple downsampling stages. Each downsampling stage reduces spatial resolution and increases channel dimension through convolution operations to extract high-level semantic features. The decoder is defined as a symmetrical path consisting of multiple upsampling stages. Each upsampling stage increases spatial resolution and reduces channel dimension through transposed convolution operations to recover detailed information. Skip connections are established between the corresponding downsampling and upsampling stages of the encoder and decoder to achieve multi-scale feature transfer, forming a U-shaped network architecture with a symmetrical topology, thus completing the construction of the topology layer.

[0080] The feature transformation layer is constructed by establishing an alternating stacking rule between the feature extraction sub-layer and the resolution transformation sub-layer: the feature extraction sub-layer is responsible for capturing local feature patterns, and the resolution transformation sub-layer is responsible for reducing or increasing the dimensionality of the feature map. The feature extraction sub-layer and the resolution transformation sub-layer are combined in a fixed order (defined based on the principle of information preservation and computational optimization, such as the feature extraction sub-layer first and the resolution transformation sub-layer later). The combination pattern is repeated to form a multi-level hierarchical architecture of progressive abstraction or progressive refinement, thus completing the construction of the feature transformation layer.

[0081] An encoder-decoder attention weight sublayer is established from the encoder path to the decoder path, and a decoder-encoder attention weight sublayer is established from the decoder path to the encoder path. A feature fusion layer is constructed through the bidirectional symmetric structure of the encoder-decoder attention weight sublayer and the decoder-encoder attention weight sublayer. The two attention weight generation sublayers independently calculate the spatial attention weight mask and apply it to the feature representation of the corresponding path through feature weighting operations. The feature output after bidirectional attention modulation is integrated through the weighted fusion sublayer to complete the construction of the feature fusion layer.

[0082] S2.2. A deep learning feature extraction network constructed based on a topology layer, a feature transformation layer, and a feature fusion layer;

[0083] It should be noted that, based on the topology layer as the basic framework, the feature transformation layer is used as the basic operation to fill the basic framework, and the feature fusion layer is embedded in the skip connection node to perform advanced feature modulation, thus completing the construction of the deep learning feature extraction network.

[0084] Defect regions are labeled on the acquired optical surface images to serve as real defect labels. Multimodal image data is used as input, and a training deep feature map is calculated through forward propagation. The loss function between the real defect labels and the training deep feature map is calculated. Gradient descent algorithm is used, and the partial derivatives of the loss function with respect to each learnable parameter (weight parameter and bias parameter) in the deep learning feature extraction network are calculated through backpropagation. The weight parameters are updated according to the gradient direction to minimize the loss function. The forward propagation, loss calculation, backpropagation, and parameter update process is iteratively repeated until the loss function converges (the rate of change of the loss function is lower than the convergence tolerance) or the preset number of training rounds (based on the definition of computational resource constraints) is reached. The optimized network weight parameters are obtained, and the training of the deep learning feature extraction network is completed.

[0085] S2.3 Input multimodal image data into a deep learning feature extraction network. The topology layer constructs encoding and decoding paths, the feature transformation layer performs feature extraction and resolution transformation, and the feature fusion layer generates a deep feature map through feature enhancement and fusion.

[0086] It should be noted that multimodal image data is input into a deep learning feature extraction network. The topology layer performs convolution operations and nonlinear transformations on the multimodal image data through the encoding path to extract basic feature patterns and generate primary feature maps. In each downsampling stage of the encoding path, convolution kernel sliding calculations are performed sequentially to capture local feature associations. Pooling operations are used to compress the spatial dimension of the primary feature map and expand the channel dimension, gradually abstracting to form high-level semantic features. In each upsampling stage of the decoding path, transposed convolution operations are used to restore the spatial resolution of the primary feature map. Combined with the corresponding level encoded features passed by skip connections, convolution calculations are performed to optimize feature representation and reconstruct detailed information. At the skip connection nodes corresponding to the encoder and decoder, a bidirectional attention mechanism is used to calculate the weight mask of the encoded features on the decoded features and the weight mask of the decoded features on the encoded features, respectively. The original features in the primary feature map are weighted, modulated, and fused to generate enhanced features. At the end of the decoding path, a deep feature map is output.

[0087] It should also be noted that existing technologies extract image features through a single convolutional path, which can capture basic texture information but are difficult to effectively integrate complementary features between multimodal data and have semantic gaps in cross-scale feature interactions. This solution solves the problem of insufficient response to weak defects on optical surfaces by constructing a deep learning feature extraction network to generate deep feature maps, providing a more robust feature representation basis for high-precision defect localization.

[0088] S3. Use deep learning region generation methods to perform spatial domain analysis on deep feature maps, locate the coordinates of potential defect regions, and generate a set of candidate defect region coordinates.

[0089] S3.1 Construct a multi-scale feature pyramid based on the depth feature map, and pre-set anchor boxes of different sizes and aspect ratios at each spatial location of the multi-scale feature pyramid to generate an initial set of anchor boxes;

[0090] It should be noted that convolution operations are performed on the depth feature map to maintain the baseline resolution. Convolution operations with a stride greater than one are used to gradually reduce the spatial resolution of the depth feature map and increase the number of channels to generate low-resolution feature maps. Simultaneously, upsampling operations are used to gradually increase the spatial resolution of the depth feature map and reduce the number of channels to generate high-resolution feature maps. Feature maps of different resolutions are arranged in scale order to form a multi-scale feature pyramid.

[0091] The scale order is defined based on the hierarchical abstraction principle in the feature pyramid theory, referring to the top-down arrangement order from high-resolution feature maps to low-resolution feature maps;

[0092] Traversing all rows and columns in each feature map of the multi-scale pyramid, each intersection point of a row and column corresponds to a spatial location, and the center coordinates of all spatial locations are the center reference position of the anchor box of each feature map layer; the different sizes and aspect ratios of the anchor boxes are based on the scale distribution and shape characteristics of optical surface defects. The size settings refer to the actual size range of optical surface defects in the image, and usually multiple incremental reference sizes are selected (such as 16x16, 32x64, 128x256, etc.); the aspect ratio is set according to the common proportions of optical surface defect shapes (such as 1:1 for square targets, 1:2 for tall and thin targets, and 2:1 for flat targets). An initial set of anchor boxes covering multiple scales and shapes is generated at the center reference position of each feature map layer in the multi-scale pyramid.

[0093] S3.2 Calculate the defect confidence score and bounding box coordinate offset of each initial anchor box based on the multi-scale feature pyramid to generate a preliminary set of candidate anchor boxes;

[0094] It should be noted that, based on the center reference position and size of the anchor box, the covered feature map range is located on the corresponding feature map of the multi-scale feature pyramid, and the feature vectors of all feature points within the feature map range are obtained; the probability value that the corresponding anchor box contains the defect target is calculated as the defect confidence score through classification operations, and the expression is as follows:

[0095] ;

[0096] in, This indicates the confidence score for the defect; This represents the summation index, used to simultaneously iterate through the corresponding elements of the feature vector and weight parameters, with a value range of 1-... ; This represents the total number of elements in the feature vector and weight parameters; Indicates the weight parameter of the first One element; Represents the eigenvector of the th One element; Indicates the bias parameter; This represents the Sigmoid activation function;

[0097] The adjustment amounts of the anchor point center point coordinates, width, and height are calculated using regression operations and used as the bounding box coordinate offsets. The expression is as follows:

[0098] ;

[0099] ;

[0100] in, This represents the bounding box coordinate offset. The variable is called the independent variable, and different values ​​correspond to different offsets. Indicates the center point of the anchor box coordinate, Indicates the center point of the anchor box coordinate, The scaling factor representing the width of the anchor box. The scaling factor representing the height of the anchor point frame;

[0101] After traversing all anchor boxes, a preliminary set of candidate anchor boxes is generated, which includes the defect confidence score and bounding box coordinate offset of each anchor box.

[0102] S3.3 Sort the preliminary candidate anchor boxes according to the defect confidence score, select the retained boxes, calculate the cross-union ratio based on the retained boxes, compare the cross-union ratio with the preset cross-union ratio threshold, and generate a refined candidate anchor box set based on the comparison results;

[0103] It should be noted that all preliminary candidate anchor boxes in the initial candidate anchor box set are sorted from highest to lowest according to their defect confidence scores. The preliminary candidate anchor box with the highest score is then selected as the retained box. The intersection-union ratio (IUU) of the remaining retained boxes with the preliminary candidate anchor boxes is calculated, expressed as follows:

[0104] ;

[0105] in, Indicates a reserved box With the initial candidate anchor box The intersection and union ratio; Indicates a reserved box; Indicates the first A preliminary candidate anchor point box; Indicates the index of the initial candidate anchor box; Indicates a reserved box With the initial candidate anchor box The area of ​​the intersecting region; Indicates the area of ​​the reserved frame; Indicates the first The area of ​​the initial candidate anchor point boxes;

[0106] Remove low-scoring initial candidate anchor boxes with an intersection-union ratio (IU) higher than the IU threshold (defined based on the balance between task characteristics and evaluation metrics, such as 0.5). Repeat the process of selecting and retaining boxes and removing low-scoring initial candidate anchor boxes until all anchor boxes have been processed, generating a refined set of candidate anchor boxes.

[0107] S3.4 Calculate the scaling ratio of the depth feature map relative to the optical surface image. Based on the scaling ratio, forward map the refined candidate anchor box set to the coordinate space of the optical surface image to generate the candidate defect region coordinate set.

[0108] It should be noted that the scaling ratio in the width direction is obtained by calculating the ratio of the width of the optical surface image to the width of the depth feature map, and the scaling ratio in the height direction is obtained by calculating the ratio of the height of the optical surface image to the height of the depth feature map. The expression is as follows:

[0109] ;

[0110] ;

[0111] in, Indicates the scaling ratio in the width direction; Indicates the width of the optical surface image; Indicates the width of the depth feature map; Indicates the scaling ratio in the height direction; Indicates the height of the optical surface image; Indicates the height of the depth feature map;

[0112] Based on the scaling ratios in the width and height directions, the coordinate data of each refined candidate anchor point bounding box in the optical surface image is calculated, including the center point coordinates, width, and height values. The coordinate data of all refined candidate anchor point bounding boxes in the optical surface image are combined to generate a set of candidate defect region coordinates. The expression for calculating the coordinate data is as follows:

[0113] ;

[0114] ;

[0115] ;

[0116] ;

[0117] in, This represents the center point of the refined candidate anchor point box on the optical surface image. coordinate; This represents the center point of the refined candidate anchor point box on the optical surface image. coordinate; This represents the width of the refined candidate anchor point box on the optical surface image; This indicates the height of the refined candidate anchor point box on the optical surface image; This represents the center point of the refined candidate anchor box on the depth feature map. coordinate; This represents the center point of the refined candidate anchor box on the depth feature map. coordinate; This represents the width of the refined candidate anchor box on the depth feature map; This represents the height of the refined candidate anchor box on the depth feature map.

[0118] S4. Based on the set of candidate defect region coordinates and the depth feature map, the local feature map corresponding to each candidate defect region is cropped from the depth feature map, and the scale of the local feature map is normalized to generate a set of candidate region feature maps.

[0119] S4.1 Based on the size scaling ratio, each anchor box in the candidate defect region coordinate set is reverse mapped to the depth feature map to generate a set of depth feature map anchor boxes;

[0120] It should be noted that, based on the scaling ratios in the width and height directions, the ratios of the x-coordinate of the center point of each anchor box in the candidate defect region coordinate set to the scaling ratio in the width direction, the y-coordinate of the center point to the scaling ratio in the height direction, the width value to the scaling ratio in the width direction, and the height value to the scaling ratio in the height direction are calculated. This generates anchor box coordinate data that is inversely mapped to the depth feature map. The coordinate transformation of the inverse mapping is completed by traversing all candidate defect region coordinates, generating a set of anchor boxes for the depth feature map.

[0121] It should also be noted that the purpose of forward mapping is to transform the detection results on the depth feature map into physical coordinates of the optical surface image that users can understand, so as to meet the need for accurate location of defects in practical applications. The coordinate set after forward mapping completely retains the candidate box information after non-maximum suppression screening. Inverse mapping is to accurately backtrack to the corresponding region of the depth feature map based on the refined physical coordinates to perform feature clipping, so as to ensure that the local features used for classification are strictly aligned with the defect position of the final output, thereby solving the feature misalignment problem caused by inconsistent coordinates.

[0122] S4.2 Perform bilinear interpolation feature clipping on each set of anchor points in the depth feature map to extract the local feature map of the corresponding region;

[0123] It should be noted that, based on the center point coordinates and size parameters of each anchor point in the depth feature map anchor box set, a corresponding rectangular region is located on the depth feature map. The feature values ​​of each sampling point within this rectangular region are calculated using bilinear interpolation: The four nearest integer coordinate sampling points (bottom left, bottom right, top left, and top right) of the sampling point are located based on its continuous coordinates. The horizontal distance difference between the sampling point and the sampling point to its left, and the vertical distance difference with the sampling point below are calculated respectively. The ratio of the horizontal distance difference to the horizontal spacing of the sampling points is used as the horizontal weight coefficient, and the ratio of the vertical distance difference to the vertical spacing of the sampling points is used as the vertical weight coefficient. Linear interpolation is performed on the four sampling points in the horizontal direction based on the horizontal weight coefficient. A second linear interpolation is performed on the horizontal interpolation result based on the vertical weight coefficient to obtain the feature value of the target sampling point. The expression for bilinear interpolation is as follows:

[0124] ;

[0125] in, Indicates sampling point eigenvalues; Indicates the horizontal weighting coefficient; Indicates the vertical weighting coefficient; Indicates the sampling point in the lower left corner. eigenvalues; Indicates the sampling point in the lower right corner. eigenvalues; Indicates the sampling point in the upper left corner. eigenvalues; Indicates the sampling point in the upper right corner. eigenvalues;

[0126] Iterate through the feature values ​​of all sampling points within the rectangular region to generate a local feature map that matches the anchor box size of the depth feature map; repeat the bilinear interpolation operation on all sampling points within the rectangular region to generate a set of local feature maps.

[0127] S4.3. Normalize local feature maps of different sizes to a fixed size using bilinear interpolation, and integrate all local feature maps of fixed sizes in the same order in the candidate defect region coordinate set to generate a candidate region feature map set.

[0128] It should be noted that, based on task requirements and network structure design, the target size (e.g., 7×7) is defined, and the target size is divided into a target size grid (a uniform two-dimensional coordinate point matrix). Each target size grid point corresponds to the position coordinates on the target feature map (referring to a local feature map of a uniform and fixed size).

[0129] Bilinear interpolation is used to scale local feature maps of different sizes to a target-size grid. Specifically, the continuous coordinates of each sampling point in the target-size grid within the local feature map are calculated, as shown in the expression.

[0130] ;

[0131] ;

[0132] in, This represents the continuous horizontal coordinates mapped to the local feature map; This represents the continuous vertical coordinates mapped to the local feature map; This represents the actual width of the local feature map; This represents the actual height of the local feature map; This indicates the horizontal coordinate of the current sampling point within the target-size grid. This indicates the vertical coordinate of the current sampling point within the target-size grid. Indicates the target dimension width; Indicates the target dimension height; The parameter category representing the local feature map; The parameter category indicating the target size;

[0133] The feature value of the target sampling point is obtained by calculating the distance-weighted average of the feature values ​​of the four nearest sampling points around the continuous coordinate position. After completing the bilinear interpolation calculation by traversing all grid points in the target size grid, a local feature map of fixed size is generated. All local feature maps of fixed size are arranged and integrated according to the original order of the anchor point boxes in the candidate defect area coordinate set to generate a set of candidate area feature maps with consistent channel number and uniform spatial size.

[0134] S5. Compress each feature map in the candidate region feature map set into a feature vector, and extract semantic information from the feature vector through nonlinear transformation and feature dimensionality reduction to generate a candidate region feature vector set.

[0135] S5.1. Compress each feature map in the candidate region feature map set into a feature vector through global average pooling operation to generate an initial feature vector set;

[0136] It should be noted that, through global average pooling, the feature values ​​of each candidate region feature map in the candidate region feature map set at all spatial locations in each channel are summed, and the ratio of the summation result to the total number of spatial locations in the current channel is calculated to obtain the average feature value of each channel. The average feature values ​​of all channels are then concatenated and combined into a one-dimensional vector in channel order. For each candidate region feature map in the candidate region feature map set, global average pooling is performed independently, and the one-dimensional vectors of each candidate region feature map are combined to generate an initial feature vector set.

[0137] It should also be noted that channels are an inherent property of deep learning feature extraction networks when generating deep feature maps. They represent different feature types and refer to the number of feature values ​​contained in each spatial location of the feature map.

[0138] S5.2 Project each initial feature vector in the initial feature vector set into a high-dimensional space and apply the ReLU activation function to generate a high-dimensional feature vector;

[0139] It should be noted that the high-dimensional weight matrix is ​​defined based on the relationship between the length of the initial input feature vector and the target length of the high-dimensional feature vector (predefined based on task requirements and computational resource constraints). Specifically, the number of rows in the high-dimensional weight matrix is ​​set according to the number of elements in the initial input feature vector, and the number of columns in the high-dimensional weight matrix is ​​set according to the number of elements in the output high-dimensional feature vector, thus constructing a high-dimensional weight matrix with a specific number of rows and columns. A random number generator is used to sample values ​​from a uniform distribution to fill the elements of the high-dimensional weight matrix.

[0140] Each element of the initial feature vector is multiplied by each element of the corresponding row of the high-dimensional weight matrix. The sum of all products row-wise yields the value of each element of the projection vector, thus mapping the initial feature vector to a new dimensional space to generate the projection vector. The ReLU activation function is then applied to each element of the projection vector to generate the high-dimensional feature vector. The expression for the ReLU activation function is:

[0141] ;

[0142] in, Represents the ReLU activation function; This represents the input value, i.e., the value of each element of the projection vector. Each element of the projection vector is checked one by one. If the current element value is less than zero, the current element value is set to zero. If the current element value is greater than or equal to zero, the current element value is left unchanged.

[0143] S5.3 Project the high-dimensional feature vectors onto the low-dimensional space to generate dimensionality-reduced feature vectors;

[0144] It should be noted that the dimensionality reduction weight matrix is ​​defined based on the relationship between the length of the input high-dimensional feature vector and the target length of the output dimensionality reduction feature vector. Specifically, the number of rows of the dimensionality reduction weight matrix is ​​set according to the number of elements in the input high-dimensional feature vector, and the number of columns of the dimensionality reduction weight matrix is ​​set according to the number of elements in the output dimensionality reduction feature vector. The number of columns is less than the number of rows to achieve the dimensionality reduction target. A dimensionality reduction weight matrix with more rows than columns is constructed, and values ​​are sampled from a uniform distribution using a random number generator to fill the elements of the dimensionality reduction weight matrix.

[0145] Each element of the high-dimensional feature vector is multiplied by each element in the corresponding row of the dimensionality reduction weight matrix. All product results are summed row by row to obtain the value of each element of the dimensionality reduction feature vector. Since the number of columns in the dimensionality reduction weight matrix is ​​small, the dimensionality of the high-dimensional feature vector is reduced, thus realizing the mapping of the high-dimensional feature vector to the low-dimensional space to generate the dimensionality reduction feature vector.

[0146] S5.4 Perform batch normalization on the dimensionality-reduced feature vectors to generate standardized feature vectors;

[0147] It should be noted that for each batch of dimensionality-reduced feature vectors, the mean and variance of each feature dimension are calculated. The expressions for calculating the mean and variance are as follows:

[0148] ;

[0149] ;

[0150] in, Indicates the first The mean of each feature dimension; Indicates the first The variance of each feature dimension; This indicates the number of samples in the batch, i.e., the number of dimensionality-reduced feature vectors in the batch; Indicates the first The sample at the th Values ​​in each feature dimension; This represents the sample index, with a value range of 1- ;

[0151] The mean is calculated as the difference between the mean and the eigenvalues ​​in each dimensionality-reduced eigenvector to obtain the centered eigenvalues; the variance and a minimum constant are calculated (based on numerical stability requirements and the definition of floating-point precision, to prevent division by zero errors when the variance is zero, such as 10). -5 The square root of the sum of the eigenvalues ​​is calculated, and the ratio of the centered eigenvalues ​​to the square root of the sum is calculated to obtain the normalized eigenvalues. The product of the normalized eigenvalues ​​and the scaling parameter (defined based on the flexibility requirements of the feature distribution) is calculated, and the product result is summed with the offset parameter (defined based on the data center recovery requirements) to generate the normalized feature vector.

[0152] S5.5 Sort the standardized feature vectors according to the original order of the candidate region feature map set to generate a candidate region feature vector set.

[0153] It should be noted that the standardized feature vectors are arranged in the original order of the corresponding anchor boxes in the candidate region feature map set, ensuring that each standardized feature vector corresponds one-to-one with the spatial position in the candidate region feature map set, and generating a candidate region feature vector set with the same order as the candidate region feature map set.

[0154] It should also be noted that existing technologies directly compress candidate region feature maps into feature vectors through global average pooling operations. While this reduces computational complexity and preserves channel semantic information, it lacks the ability to deeply explore nonlinear relationships between features and focus information, resulting in insufficient semantic expression of weak defects on optical surfaces. This solution enhances the nonlinear expression of features through high-dimensional projection and ReLU activation, then extracts key semantic information through dimensionality reduction projection, and finally performs batch normalization to stabilize the feature distribution, generating standardized feature vectors with high discriminative power. This solves the problem of missed detection of weak defects caused by insufficient feature information density.

[0155] S6. Input the generated candidate region feature vector set into the deep learning classifier, and output the defect category determination result and confidence score.

[0156] S6.1 Input each candidate region feature vector in the candidate region feature vector set into the fully connected layer for calculation to generate an initial score vector for each defect category;

[0157] It should be noted that, based on the length of the input candidate region feature vector and the number of output defect categories (defined according to the actual physical defect types and detection task requirements, such as the number of categories like scratches and stains), the number of rows in the fully connected layer weight matrix is ​​determined to be the number of defect categories, and the number of columns is the feature dimension of the input candidate region feature vector. Each row and column position stores an adjustable value representing the strength of the association between the feature and the category. All adjustable values ​​are initialized using a random number generator to form a fully connected layer weight matrix composed of row and column structures, with each position containing a specific value.

[0158] Each element of each candidate region feature vector in the candidate region feature vector set is multiplied with the corresponding element in each column of the fully connected layer weight matrix. The product results of each column are summed to obtain the initial score of the defect category corresponding to the current column. The initial scores of all defect categories are combined to generate the initial score vector.

[0159] S6.2. Convert the initial score vector into a probability distribution vector using the Softmax function to generate the probability value of each candidate region belonging to each defect category;

[0160] It should be noted that the initial score of each defect category in the initial score vector is exponentially calculated using the Softmax function to obtain the exponential score value. The sum of the exponential score values ​​of all defect categories is calculated, and the ratio of the exponential score value of each defect category to the sum of the exponential score values ​​is used as the normalized probability value. The normalized probability values ​​of all defect categories are combined in order of category to generate a probability distribution vector.

[0161] The mathematical expression for the Softmax function is:

[0162] ;

[0163] in, Indicates the first Normalized probability values ​​for each defect category; Indicates the first Initial scores for each defect category; Indicates the total number of defect categories; This represents the category index, with a value range of 1- ; Indicates the first The initial scores for each defect category are used for exponential calculation; The summation index is a loop variable that takes values ​​from 1 to... ; This indicates that in a loop summation operation, the first... Initial scores for each defect category.

[0164] S6.3 Extract the defect category corresponding to the maximum probability value as the defect category determination result, and output the maximum probability value as the confidence score.

[0165] It should be noted that all defect categories that need to be identified are clearly defined and a name list is formed. Each defect category is assigned a continuously increasing integer number starting from zero according to the order of the name list, forming a static lookup table that binds the number to the defect category.

[0166] The probability values ​​of all defect categories in the probability distribution vector are iterated and compared to find the maximum probability value and the corresponding defect category index. The defect category index is input into a static lookup table to find the corresponding actual defect type name as the defect category determination result, and the maximum probability value is directly used as the confidence score output.

[0167] This embodiment also provides a computer device applicable to the deep learning-based optical surface defect data detection method, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the deep learning-based optical surface defect data detection method proposed in the above embodiment.

[0168] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0169] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the deep learning-based optical surface defect data detection method proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0170] In summary, this invention achieves a substantial breakthrough in detection accuracy and robustness by: using multimodal data synthesis driven by an optical scattering physics model to overcome the limitations of a single imaging mode; dynamically analyzing the scattering characteristics of defects under multi-physics coupling to enhance the identifiability of weak defects in complex scattering environments; establishing symmetrical interaction between the encoder and decoder paths through a bidirectional attention feedback mechanism; and fusing cross-level features through bidirectional weight modulation to coordinate the consistent expression of local fine textures and global structural semantics, thereby eliminating the redundancy in the location of irregular composite defects and fluctuations in classification confidence.

[0171] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for detecting optical surface defect data based on deep learning, characterized in that: include, The specific steps for constructing a physical model of optical scattering are as follows. A theoretical framework layer is constructed based on the existing bidirectional reflection distribution function theory, a physical property calculation layer is constructed based on the light scattering characteristics of microsurfaces, and a parameter configuration layer is constructed based on the physical property parameter types. An optical scattering physics model is constructed by combining the theoretical framework layer, the physical property calculation layer, and the parameter configuration layer. The acquired optical surface images are input into the optical scattering physics model for multimodal data synthesis to generate multimodal image data. The specific steps are as follows. Optical surface images are acquired using an image sensor and a controllable illumination structure, and the images are analyzed using an inverse rendering method to extract physical property parameter values. The physical property parameter values ​​are assigned to the parameter configuration layer, the values ​​of the parameter configuration layer are called according to the rules of the theoretical framework layer, Monte Carlo rendering is performed in the physical property calculation layer, and a composite image under the current lighting conditions is generated. Multimodal image data is generated after traversing all preset lighting conditions; The specific steps for constructing a deep learning feature extraction network are as follows. A topology layer is constructed based on an encoder-decoder architecture, a feature transformation layer is constructed based on feature extraction and resolution transformation operations, and a feature fusion layer is constructed based on a bidirectional attention feedback mechanism. A deep learning feature extraction network is constructed based on a topology layer, a feature transformation layer, and a feature fusion layer. The multimodal image data is then input, and a bidirectional attention feedback mechanism is used to perform multi-scale feature fusion and enhancement to generate a deep feature map. The specific steps are as follows: Multimodal image data is input into a deep learning feature extraction network, and the multimodal image data is encoded and decoded through a topological structure layer; The feature transformation layer performs feature extraction and resolution transformation, while the feature fusion layer generates a depth feature map through feature enhancement and fusion. A deep learning-based region generation method is used to perform spatial domain analysis on the deep feature map, locate the coordinates of potential defect regions, and generate a set of candidate defect region coordinates. The specific steps are as follows. A multi-scale feature pyramid is constructed based on the deep feature map. An anchor box with different size and aspect ratio is preset at each spatial location of the multi-scale feature pyramid to generate an initial set of anchor boxes. Based on the multi-scale feature pyramid, the defect confidence score and bounding box coordinate offset of each initial anchor box are calculated to generate a preliminary set of candidate anchor boxes; The preliminary candidate anchor boxes are sorted according to the defect confidence score, the selected boxes are retained, and the cross-union ratio is calculated based on the retained boxes. The preliminary candidate anchor boxes whose cross-union ratio does not exceed the preset cross-union ratio threshold are extracted to generate a refined candidate anchor box set. Calculate the scaling ratio of the depth feature map relative to the optical surface image, and based on the scaling ratio, forward map the refined candidate anchor box set to the coordinate space of the optical surface image to generate the coordinate set of candidate defect regions. The local feature map corresponding to each candidate defect region is cropped from the depth feature map, and the scale of the local feature map is normalized to generate a set of candidate region feature maps. Each feature map in the candidate region feature map set is compressed into a feature vector, and semantic information is extracted from the feature vector through nonlinear transformation and feature dimensionality reduction to generate a candidate region feature vector set. The set of feature vectors of candidate regions is input into a deep learning classifier, which outputs the defect category determination result and confidence score.

2. The method for detecting optical surface defects based on deep learning as described in claim 1, characterized in that: The specific steps for cropping the local feature map corresponding to each candidate defect region from the depth feature map are as follows: Based on the size scaling ratio, each anchor box in the candidate defect region coordinate set is inversely mapped to the depth feature map to generate a depth feature map anchor box set. Bilinear interpolation feature clipping is performed on each set of anchor points in the depth feature map to extract the local feature map of the corresponding region.

3. The method for detecting optical surface defects based on deep learning as described in claim 2, characterized in that: The steps for scaling the local feature maps to generate a set of candidate region feature maps are as follows. Local feature maps of different sizes are normalized to a fixed size using bilinear interpolation; All fixed-size local feature maps are integrated in the same order as the candidate defect region coordinate set to generate a candidate region feature map set.

4. The method for detecting optical surface defects based on deep learning as described in claim 3, characterized in that: The specific steps for creating the candidate region feature vector set are as follows. The initial feature vector set is generated by compressing each feature map in the candidate region feature map set into an initial feature vector through a global average pooling operation. Each initial feature vector in the initial feature vector set is projected into a high-dimensional space and the ReLU activation function is applied to generate a high-dimensional feature vector. Projecting high-dimensional feature vectors onto a low-dimensional space generates dimensionality-reduced feature vectors. Batch normalization is performed on the dimensionality-reduced feature vectors to generate standardized feature vectors; The standardized feature vectors are sorted according to the original order of the candidate region feature map set to generate a candidate region feature vector set.

5. The method for detecting optical surface defects based on deep learning as described in claim 4, characterized in that: The specific steps for inputting the candidate region feature vector set into the deep learning classifier and outputting the defect category determination result and confidence score are as follows. Each candidate region feature vector in the candidate region feature vector set is input into the fully connected layer for calculation to generate an initial score vector for each defect category; The initial score vector is converted into a probability distribution vector using the Softmax function, generating the probability value of each candidate region belonging to each defect category; Extract the defect category corresponding to the highest probability value as the defect category determination result, and use the highest probability value as the confidence score.

Images