Optical surface defect data detection method based on deep learning

Abstract

The invention discloses an optical surface defect data detection method based on deep learning, and relates to the technical field of optical defect detection, and the method comprises the following steps: constructing an optical scattering physical model, inputting a collected optical surface image into the optical scattering physical model for multi-modal data synthesis, and generating multi-modal image data; constructing a deep learning feature extraction network, inputting multi-modal image data, and performing multi-scale feature fusion and enhancement through a bidirectional attention feedback mechanism to generate a deep feature map; performing spatial domain analysis on the depth feature map by using a deep learning region generation method, positioning coordinates of potential defect regions, and generating a candidate defect region coordinate set; through multi-modal data synthesis driven by an optical scattering physical model, the limitation of a single imaging mode is broken through, the scattering characteristics of defects under multi-physics field coupling are dynamically analyzed, the recognizable degree of weak defects in a complex scattering environment is enhanced, and the problem of defect missing detection is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of optical defect detection, and in particular to an optical surface defect data detection method based on deep learning. BACKGROUND

[0002] In recent years, deep learning technology has made progress in the field of optical surface defect detection. The feature extraction method based on convolutional neural network can automatically learn the abstract representation of surface texture, replacing the limitations of traditional manual design features; the combination of region proposal network (RPN) and feature pyramid network (FPN) improves the positioning ability of multi-scale defects, and the introduction of attention mechanism further strengthens the focusing ability of weak defect features. In addition, the end-to-end detection framework improves the robustness of defect recognition in complex scattering environment by jointly optimizing the positioning and classification tasks.

[0003] The existing optical surface defect detection method has the following deficiencies: it relies on a single imaging mode and cannot dynamically analyze the multi-physical field coupling characteristics of defects, resulting in insufficient signal-to-noise ratio of weak defects under certain lighting conditions, increased missed detection rate, and a one-way feature transfer path that lacks symmetric interaction ability of cross-level features. High-level semantic features and low-level detail features are only fused through simple skip connection, without establishing a bidirectional feedback mechanism, resulting in broken context association of multi-scale defects, and one-way fusion that is difficult to coordinate consistent expression of local texture and global structure, causing redundant positioning frame or fluctuation of classification confidence. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides an optical surface defect data detection method based on deep learning to solve the problems of weak defect missed detection and insufficient cross-scale semantic fusion.

[0006] To solve the above technical problems, the present application provides the following technical solutions: The application provides a kind of based on deep learning optical surface defect data detection method, it includes, constructs optical scattering physical model, the optical surface image collected is input optical scattering physical model and carries out multimodal data synthesis, generates multimodal image data;Construct deep learning feature extraction network, and multimodal image data is input, through bidirectional attention feedback mechanism carries out multiscale feature fusion and enhancement, generates deep feature map;Depth feature map is analyzed in space domain using deep learning region generation method, the coordinates of potential defect area are located, and candidate defect area coordinate set is generated;From deep feature map, the local feature map corresponding to each candidate defect area is cropped out, and scale normalization is carried out to local feature map, and candidate area feature map set is generated;Each feature map in candidate area feature map set is compressed into feature vector, and semantic information is extracted to feature vector by nonlinear transformation and feature dimension reduction, and candidate area feature vector set is generated;Candidate area feature vector set is input deep learning classifier, and defect category determination result and confidence score are output.

[0007] As a preferred scheme of the deep learning-based optical surface defect data detection method described in the application, wherein: the optical scattering physical model is constructed, and the specific steps are as follows, The theoretical framework layer is constructed based on the bidirectional reflectance distribution function theory, the physical property calculation layer is constructed based on the micro-surface light scattering characteristics, and the parameter configuration layer is constructed based on the physical property parameter type. The optical scattering physical model is constructed in combination with the theoretical framework layer, the physical property calculation layer and the parameter configuration layer.

[0008] As a preferred scheme of the deep learning-based optical surface defect data detection method described in the application, wherein: the multimodal image data is generated, and the specific steps are as follows, The optical surface image is collected by an image sensor and a controllable lighting device, and the inverse rendering method is used to analyze the optical surface image to extract the physical property parameter values. The physical property parameter values are assigned to the parameter configuration layer, the values of the parameter configuration layer are called according to the rules of the theoretical framework layer, the Monte Carlo rendering is performed in the physical property calculation layer to generate a synthesized image under the current lighting condition, and the multimodal image data is generated after traversing all the preset lighting conditions.

[0009] As a preferred scheme of the deep learning-based optical surface defect data detection method described in the application, wherein: the deep learning feature extraction network is constructed, and the specific steps are as follows, The topology structure layer is constructed based on the encoder-decoder architecture, the feature transformation layer is constructed based on the feature extraction and resolution transformation operation, and the feature fusion layer is constructed based on the bidirectional attention feedback mechanism. The deep learning feature extraction network is constructed based on the topology structure layer, the feature transformation layer and the feature fusion layer.

[0010] As a preferred scheme of the optical surface defect data detection method based on deep learning provided in the application, the generation of the deep feature map comprises the following steps, The multi-modal image data is input into the deep learning feature extraction network, and the multi-modal image data is encoded and decoded through the topological structure layer; The feature transformation layer performs feature extraction and resolution transformation, and the feature fusion layer performs feature enhancement and fusion to generate the deep feature map.

[0011] As a preferred scheme of the optical surface defect data detection method based on deep learning provided in the application, the generation of the candidate defect region coordinate set comprises the following steps, A multi-scale feature pyramid is constructed based on the deep feature map, anchor boxes of different sizes and aspect ratios are preset at each spatial position of the multi-scale feature pyramid, and an initial anchor box set is generated; Based on the multi-scale feature pyramid, the defect confidence score and the boundary box coordinate offset of each initial anchor box are calculated to generate a preliminary candidate anchor box set; The preliminary candidate anchor boxes are sorted according to the defect confidence scores, the reserved boxes are selected, and the intersection over union is calculated based on the reserved boxes. The preliminary candidate anchor boxes whose intersection over union does not exceed the preset intersection over union threshold are extracted to generate a refined candidate anchor box set; The size scaling ratio of the deep feature map relative to the optical surface image is calculated, and the refined candidate anchor box set is mapped to the coordinate space of the optical surface image in a forward direction based on the size scaling ratio to generate the candidate defect region coordinate set.

[0012] As a preferred scheme of the optical surface defect data detection method based on deep learning provided in the application, the generation of the candidate defect region coordinate set and the deep feature map, and the cutting of the local feature map corresponding to each candidate defect region from the deep feature map comprises the following steps, Each anchor box in the candidate defect region coordinate set is inversely mapped to the deep feature map based on the size scaling ratio to generate a deep feature map anchor box set; Bilinear interpolation feature cutting is performed on each deep feature map anchor box set on the deep feature map to extract the local feature map of the corresponding region.

[0013] As a preferred scheme of the optical surface defect data detection method based on deep learning provided in the application, the scale normalization of the local feature map to generate the candidate region feature map set comprises the following steps, Different sizes of the local feature map are normalized to a fixed size by bilinear interpolation; Integrate all fixed-size local feature maps in the same order in the candidate defect region coordinate set to generate a candidate region feature map set.

[0014] As a preferred scheme of the optical surface defect data detection method based on deep learning, wherein the candidate region feature vector set is specifically as follows, Compress each feature map in the candidate region feature map set into an initial feature vector through a global average pooling operation to generate an initial feature vector set. Project each initial feature vector in the initial feature vector set into a high-dimensional space and apply a ReLU activation function to generate a high-dimensional feature vector. Project the high-dimensional feature vector into a low-dimensional space to generate a reduced dimension feature vector. Perform batch normalization processing on the reduced dimension feature vector to generate a standardized feature vector. Sort the standardized feature vector in the original order of the candidate region feature map set to generate a candidate region feature vector set.

[0015] As a preferred scheme of the optical surface defect data detection method based on deep learning, wherein the candidate region feature vector set is specifically as follows, Input each candidate region feature vector in the candidate region feature vector set into a fully connected layer for calculation to generate an initial score vector of each defect category. Convert the initial score vector into a probability distribution vector through a Softmax function to generate a probability value of each candidate region belonging to each defect category. Extract the defect category corresponding to the maximum probability value as the defect category determination result, and take the maximum probability value as the confidence score.

[0016] The present application has the following advantages: through multi-modal data synthesis driven by an optical scattering physical model, the limitations of a single imaging mode are broken through, the scattering characteristics of defects under multi-physical field coupling are dynamically analyzed, the recognizability of weak defects in a complex scattering environment is enhanced, and the defect missing detection problem is solved; through a bidirectional attention feedback mechanism, a symmetric interaction between an encoder and a decoder path is established, cross-level features are fused through bidirectional weight modulation, the consistency expression of local subtle texture and global structure semantics is coordinated, the positioning redundancy of irregular composite defects and the classification confidence fluctuation are eliminated, and substantial breakthroughs in detection accuracy and robustness are achieved. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description only constitute some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort.

[0018] Fig. 1 Flowchart for the optical surface defect data detection method based on deep learning.

[0019] Fig. 2 Flowchart for generating multi-modal image data.

[0020] Fig. 3 Flowchart for generating deep feature maps.

[0021] Fig. 4 Flowchart for outputting defect category determination results and confidence scores. DETAILED DESCRIPTION

[0022] In order to make the above objectives, features and advantages of the present application more apparent and comprehensible, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0023] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application. However, the present application can be practiced without the specific details, other than those described herein, and it will be apparent to those skilled in the art that the present application can be practiced with other than the described embodiments, which have been presented for purposes of example and illustration and not limitation, unless otherwise specifically indicated.

[0024] Secondly, the "one embodiment" or "embodiment" referred to herein can include specific features, structures or characteristics contained in at least one implementation of the present application. "In one embodiment" appearing in different places in the specification does not mean the same embodiment, nor is it an independent or alternative embodiment that excludes other embodiments.

[0025] Reference Figs. 1-4 For one embodiment of the present application, the embodiment provides an optical surface defect data detection method based on deep learning, comprising the following steps: S1, constructing an optical scattering physical model, inputting the collected optical surface image into the optical scattering physical model for multi-modal data synthesis to generate multi-modal image data; S1.1, constructing a theoretical framework layer based on the bidirectional reflectance distribution function theory, constructing a physical property calculation layer based on the micro-surface light scattering characteristics, and constructing a parameter configuration layer based on the physical property parameter types; It should be noted that the mathematical expression of the bidirectional reflectance distribution function is used as the mathematical form for light scattering calculations. The bidirectional reflectance distribution function defines the radiative energy mapping relationship between the incident and emitted light. Based on the principle of energy conservation, the bidirectional reflectance distribution function is required to satisfy integral constraints, and the Helmholtz reciprocity principle is followed to ensure mathematical symmetry. Based on the bidirectional reflectance distribution function, a rendering equation is defined, completing the construction of the theoretical framework. The mathematical expression of the bidirectional reflectance distribution function is: ; = ; in, The radiance produced per unit irradiance refers to the radiance produced at a wavelength of [wavelength missing]. Below, the surface material will change from a unit vector of Light incident on the direction of incidence is reflected into a unit vector of... The efficiency of the emission direction, that is, how the surface material converts the incident light energy (irradiance) into the emitted light brightness (radiance). Indicates the direction of light incidence. , unit vector; Indicates the direction of light emission. , unit vector; Indicates wavelength; Indicates the emitted radiance; Indicates incident irradiance; Indicates the incident radiation brightness; This indicates the angle of incidence, which is the angle between the incident light and the surface normal. Indicates the direction of incidence; Indicates the direction of launch; The expression for the rendering equation is, ; in, Indicates the direction of the incident light. and the direction of the emitted light The emitted radiation brightness; This represents the surface roughness parameter; Indicates the basic reflectivity parameter; Represents the distribution function of the normal to the micro-surface; ; Represents the Fresnel reflectivity function; The bidirectional reflectance distribution function defines the physical principle of light-surface interaction, quantifies the efficiency of a surface material in converting incident light energy into outgoing radiance at a specific wavelength, reflects the radiative energy mapping relationship of light scattering, and follows the principle of energy conservation and the Helmholtz reciprocity principle to ensure physical correctness. The micro-surface normal distribution function is used to describe the directional influence of the micro-surface geometry on light scattering, a geometric attenuation function is introduced to represent the energy attenuation caused by the blocking effect between micro-surfaces, and the Fresnel reflectivity function is integrated to calculate the change rule of the material reflectivity with the incident angle. The micro-surface normal distribution function, the geometric attenuation function and the Fresnel reflectivity function are integrated through the mathematical framework of the bidirectional reflectance distribution function to form a light scattering attribute calculation system, and the scattering efficiency coefficient is output to complete the construction of the physical attribute calculation layer. The mathematical expression of the scattering efficiency coefficient is, ; ; wherein, represents the scattering efficiency coefficient; represents a half-angle vector, that is, a unit vector in the direction of the angle bisector of the incident light direction and the exit light direction; refers to the statistical distribution density of the micro-surface normal in the direction of the half-angle vector , which determines the shape and sharpness of highlights; refers to the light energy attenuation caused by the blocking effect (shadow and shielding) of the micro-surface; refers to the change rule of the material reflectivity when the incident angle of the light changes; represents a macro-surface normal unit vector; The physical attribute parameter types include surface geometry attribute parameters (surface roughness parameters), material optical attribute parameters (refractive index parameters and basic reflectivity parameters), and material classification attribute parameters (metallicity parameters); The surface roughness parameter is defined to describe the irregularity of the micro-surface geometry, the refractive index parameter is defined to represent the optical density characteristics of the material, the metallicity parameter is introduced to distinguish the reflection characteristics of conductors and dielectrics, and the basic reflectivity parameter is introduced to quantify the reflection ability of the material at the vertical incidence. A standardized data storage interface and access protocol is created for the surface roughness parameter, the refractive index parameter, the metallicity parameter and the basic reflectivity parameter to form a parameter configuration system and complete the construction of the parameter configuration layer.

[0026] S1.2, based on the theoretical framework layer, the physical attribute calculation layer and the parameter configuration layer, constructing an optical scattering physical model; It should be noted that the physical property parameters such as the surface roughness parameter, the refractive index parameter, the metallicity parameter and the base reflectivity parameter are provided as input data by the parameter configuration layer, the mathematical expression of the bidirectional reflectance distribution function and the constraint condition of the principle of energy conservation and reciprocity are provided by the theoretical framework layer, and the micro-surface normal distribution, the geometric attenuation and the Fresnel reflectivity are calculated by using the physical property parameter types of the parameter configuration layer and the mathematical expression of the bidirectional reflectance distribution function of the theoretical framework layer; the three levels are closely coupled through the dependent relationship to complete the construction of the optical scattering physical model; The optical scattering physical model itself does not need to be trained. The essence of the optical scattering physical model is a deterministic mathematical model based on physical optical laws. All physical property parameter types required by the parameter configuration layer can be obtained by extracting the collected optical surface image, rather than trainable weights learned from data by gradient descent and other optimization algorithms. Therefore, the establishment of the optical scattering physical model is the determination and assignment of the physical property parameter types. Once the physical property parameter types are determined, the calculation result of the optical scattering physical model is determined by these physical property parameters and fixed mathematical formulas. There is an explicit and derivable physical causal relationship between the output and the input of the optical scattering physical model, and there is no need to train by fitting a large amount of data.

[0027] S1.3, collecting optical surface images by using an image sensor and a controllable lighting device, and analyzing the optical surface images by using an inverse rendering method to extract physical property parameter values; It should be noted that different illumination conditions are generated by actively adjusting the geometric angle, spectral composition and polarization state of the illumination by the controllable lighting device, including illumination incident angle change (such as vertical incident bright field illumination and oblique incident dark field illumination), illumination mode distinction (such as ring light and coaxial light), spectral band switching (such as different wavelength monochromatic light illumination) and polarization state adjustment (such as linear polarization and circular polarization); An optical surface image sequence under different illumination conditions is collected by using an image sensor and a controllable lighting device, and initial estimated values are assigned to the surface roughness parameter, the refractive index parameter, the metallicity parameter and the base reflectivity parameter. The initial estimated value of the surface roughness parameter is defined based on the commonly used reference median value of the root mean square deviation of height distribution in surface topography statistics (such as 0.5). The initial estimated value of the refractive index parameter is defined based on the inherent refractive characteristic constant of optical glass materials in the visible light band (such as 1.5). The initial estimated value of the metallicity parameter is defined based on the boundary condition theory of electromagnetic wave propagation at the interface of dielectric (such as 0 or 1, which is a binary description, 0 represents a non-conductor and 1 represents a conductor). The base reflectivity parameter is defined based on the benchmark value of the reflectivity of dielectric materials in the visible light band under normal incidence (such as 0.4); For each preset lighting condition and the surface point corresponding to each pixel position of the image sensor, the initial estimated value of each physical property parameter and the scattering efficiency coefficient are substituted into the rendering equation to calculate the exit radiation brightness value of the surface point under the current lighting direction and observation direction. The calculation result of each pixel position is directly used as the pixel brightness value of the corresponding coordinate position in the simulation image. The calculation results of all pixels under each independent lighting condition constitute a complete simulation image. The simulation images corresponding to all lighting conditions are organized in sequence to form a simulation image sequence. The difference square sum of the exit radiation brightness value of each pixel position between the simulation image sequence and the collected optical surface image is used as the overall difference measure. The gradient descent algorithm is used to calculate the partial derivative of the overall difference measure with respect to the initial estimated value of the physical property parameter, and the initial estimated value of the physical property parameter is updated in the opposite direction of the gradient. The above process of synthesizing the simulation image sequence, calculating the overall difference measure, and updating the initial estimated value of the physical property parameter is repeated until the overall difference measure is lower than the convergence tolerance (defined based on the iteration termination criterion in the numerical optimization theory, such as 10 -6 ), and the extraction of the physical property parameter value is completed.

[0028] S1.4, assign the physical property parameter value to the parameter configuration layer, call the value of the parameter configuration layer according to the rules of the theoretical framework layer, perform Monte Carlo rendering in the physical property calculation layer, and generate a synthetic image under the current lighting condition; after traversing all the preset lighting conditions, generate multi-modal image data.

[0029] It should be noted that assigning the physical property parameter value to the parameter type storage location corresponding to the parameter configuration layer, according to the bidirectional reflectance distribution function mathematical expression and the energy conservation constraint rule defined by the theoretical framework layer, calling the physical property parameter value from the parameter configuration layer to the physical property calculation layer, randomly generating a large number of incident light direction samples for each pixel position of the surface point in the physical property calculation layer, substituting each incident light direction sample and the scattering efficiency coefficient into the rendering equation to generate an exit radiation brightness estimate value, and after traversing all the surface points to complete random sampling and substitution into the rendering equation calculation, all the generated exit radiation brightness estimate values are integrated into a synthetic image corresponding to the current specific lighting condition; traverse all the lighting conditions and repeat the above assignment, value calling and rendering calculation process to generate multi-modal image data containing all synthetic images under different lighting conditions.

[0030] It should also be noted that the prior art uses fixed algorithms to encrypt optical surface images, which can ensure data security but lacks dynamic analysis capability for material optical properties, and cannot adapt to the complex scattering characteristics of optical surface defects. The present scheme realizes accurate modeling and multi-modal data synthesis of the physical properties of optical surface defects by constructing an optical scattering physical model, solving the problem that encryption methods cannot capture the dynamic scattering characteristics of optical surfaces.

[0031] S2, construct a deep learning feature extraction network, and input the multi-modal image data, perform multi-scale feature fusion and enhancement through a bidirectional attention feedback mechanism, and generate a deep feature map; S2.1, construct a topology structure layer based on an encoder-decoder architecture, construct a feature transformation layer based on feature extraction and resolution transformation operations, and construct a feature fusion layer based on a bidirectional attention feedback mechanism; It should be noted that the encoder part is defined as a path composed of multiple down-sampling stages, each of which reduces the spatial resolution and increases the channel dimension through convolution operation to extract high-level semantic features; the decoder part is defined as a symmetric path composed of multiple up-sampling stages, each of which increases the spatial resolution and reduces the channel dimension through transposed convolution operation to restore the detail information; a skip connection is established between the corresponding down-sampling stage and the up-sampling stage of the encoder and the decoder to realize multi-scale feature transmission, forming a U-shaped network architecture with symmetric topology structure, and completing the construction of the topology structure layer; The feature transformation layer is constructed by establishing an alternating stacking rule of the feature extraction sublayer and the resolution transformation sublayer: the feature extraction sublayer is responsible for capturing local feature patterns, the resolution transformation sublayer is responsible for dimension reduction or dimension increase of the feature map scale, and the feature extraction sublayer and the resolution transformation sublayer are combined through a fixed sequence (defined based on the principles of information preservation and calculation optimization, such as the feature extraction sublayer first and the resolution transformation sublayer later), the repeated combination mode forms a multi-level hierarchical architecture of step-by-step abstraction or step-by-step refinement, and the construction of the feature transformation layer is completed; An encoding-decoding attention weight sublayer is established from the encoder path to the decoder path, and a decoding-encoding attention weight sublayer is established from the decoder path to the encoder path, and the feature fusion layer is constructed through the bidirectional symmetric structure of the encoding-decoding attention weight sublayer and the decoding-encoding attention weight sublayer; two attention weight generation sublayers independently calculate spatial attention weight masks and act on the feature expression of the corresponding path through feature weighting operation, and the feature output after bidirectional attention modulation is integrated through a weighted fusion sublayer, and the construction of the feature fusion layer is completed.

[0032] S2.2, a deep learning feature extraction network constructed based on the topology structure layer, the feature transformation layer and the feature fusion layer; It should be noted that the topology structure layer is used as the basic framework, the feature transformation layer is filled into the basic framework as the basic operation, the feature fusion layer is embedded at the skip connection node for high-level feature modulation, and the construction of the deep learning feature extraction network is completed; The collected optical surface image is marked with a defect area as a real defect label; the multi-modal image data is input as an input, and a training deep feature map is calculated by forward propagation; the loss function value between the real defect label and the training deep feature map is calculated, the gradient descent algorithm is used, and the partial derivative of each learnable parameter (weight parameter and bias parameter) in the deep learning feature extraction network is calculated by back propagation; the weight parameters are updated according to the gradient direction to minimize the loss function, and the forward propagation, loss calculation, back propagation and parameter update process are repeated iteratively until the loss function value converges (the change rate of the loss function value is lower than the convergence tolerance) or the preset training round is reached (defined based on the calculation resource constraint), and the optimized network weight parameters are obtained, and the training of the deep learning feature extraction network is completed.

[0033] S2.3, input the multi-modal image data into the deep learning feature extraction network, construct the encoding and decoding paths in the topological structure layer, perform feature extraction and resolution conversion in the feature transformation layer, and generate the deep feature map through feature enhancement and fusion in the feature fusion layer.

[0034] It should be noted that the multi-modal image data is input into the deep learning feature extraction network, and the topological structure layer performs convolution operation and nonlinear transformation on the multi-modal image data through the encoding path, extracts the basic feature mode and generates the primary feature map; at each down-sampling stage of the encoding path, the local feature correlation is captured by sequentially performing convolution kernel sliding calculation, and the spatial dimension of the primary feature map is compressed and the channel dimension is expanded through the pooling operation, and high-level semantic features are gradually abstracted; at each up-sampling stage of the decoding path, the spatial resolution of the primary feature map is restored through the transpose convolution operation, the corresponding level encoding features are transmitted through the jump connection, the convolution calculation is performed to optimize the feature expression and reconstruct the detail information; at the jump connection nodes corresponding to the encoder and the decoder, the weight mask of the encoding features to the decoding features and the weight mask of the decoding features to the encoding features are calculated through the bidirectional attention mechanism respectively, and the original features in the primary feature map are weighted and modulated to generate enhanced features; the deep feature map is output at the end of the decoding path.

[0035] It should also be noted that the prior art extracts image features through a single convolution path, which can capture basic texture information but cannot effectively fuse complementary features between multi-modal data, and there is a semantic gap in cross-scale feature interaction; the present scheme solves the problem of insufficient response of optical surface weak defects by constructing a deep learning feature extraction network to generate a deep feature map, providing a more robust feature expression basis for high-precision defect positioning.

[0036] S3, using a deep learning region generation method to analyze the deep feature map in the spatial domain, locating the coordinates of the potential defect region, and generating a candidate defect region coordinate set; S3.1, constructing a multi-scale feature pyramid based on the deep feature map, presetting anchor boxes of different sizes and aspect ratios at each spatial position of the multi-scale feature pyramid to generate an initial anchor box set; It should be noted that the convolution operation performed on the deep feature map maintains the reference resolution, and the spatial resolution of the deep feature map is gradually reduced and the number of channels is gradually increased to generate a low-resolution feature map through a convolution operation with a step greater than one, and at the same time, the spatial resolution of the deep feature map is gradually increased and the number of channels is gradually reduced to generate a high-resolution feature map through an upsampling operation. The feature maps of different resolutions are arranged in scale order to form a multi-scale feature pyramid; The scale order is defined based on the hierarchical abstraction principle in the feature pyramid theory, which refers to the top-down arrangement order from high-resolution feature maps to low-resolution feature maps; All rows and columns in each layer of the multi-scale pyramid are traversed, and each intersection of a row and a column corresponds to a spatial position. The center coordinates of all spatial positions are the center reference positions of the anchor boxes of each layer of the feature map. Different sizes and aspect ratios of the anchor boxes are defined based on the scale distribution and shape features of the optical surface defects. The size is set by referring to the actual size range of the optical surface defects in the image, and multiple incremental reference sizes (such as 16x16, 32x64, 128x256, etc.) are usually selected. The aspect ratio is set according to the common proportion of the shape of the optical surface defects (such as 1:1 for square targets, 1:2 for tall and thin targets, and 2:1 for flat targets). An initial anchor box set covering multiple scales and multiple shapes is generated at the center reference position of each layer of the multi-scale pyramid.

[0037] S3.2, calculating the defect confidence score and boundary box coordinate offset of each initial anchor box based on the multi-scale feature pyramid to generate a preliminary candidate anchor box set; It should be noted that the feature vector of all feature points in the feature map range is obtained according to the center reference position and size of the anchor box covering the corresponding feature map in the multi-scale feature pyramid. The probability value of containing a defect target in the corresponding anchor box is calculated as the defect confidence score through the classification operation, and the expression is, ; Wherein, represents the defect confidence score; represents the summation index, which is used to traverse the corresponding elements of the feature vector and the weight parameter at the same time, and the value range is 1- ; represents the total number of elements of the feature vector and the weight parameter; represents the th element in the weight parameter; represents the th element in the feature vector; denotes a bias parameter; denotes a Sigmoid activation function; The adjustment amount of the anchor box center point coordinates, width and height is calculated as the boundary box coordinate offset through the regression operation, and the expression is, ; ; wherein, denotes the boundary box coordinate offset, is an independent variable, and different values correspond to different offsets, denotes the anchor box center point coordinates, denotes the anchor box center point coordinates, denotes the anchor box width scaling factor, denotes the anchor box height scaling factor; After traversing all the anchor boxes, a preliminary candidate anchor box set containing the defect confidence score and the boundary box coordinate offset of each anchor box is generated.

[0038] S3.3, sort the preliminary candidate anchor boxes according to the defect confidence score, select the reserved boxes, and calculate the intersection over union based on the reserved boxes, compare the intersection over union with the preset intersection over union threshold, and generate a refined candidate anchor box set according to the comparison result; It should be noted that all the preliminary candidate anchor boxes in the preliminary candidate anchor box set are sorted according to the defect confidence score from high to low, and the preliminary candidate anchor box with the highest score is selected as the reserved box, and the intersection over union of the remaining reserved boxes and the preliminary candidate anchor boxes is calculated, and the expression is, ; wherein, denotes the intersection over union of the reserved box and the preliminary candidate anchor box; denotes the reserved box; denotes the first preliminary candidate anchor box; denotes the index of the preliminary candidate anchor box; denotes the intersection over union of the reserved box and the preliminary candidate anchor box; denotes the area of the intersection region of the reserved box and the preliminary candidate anchor box; denotes the area of the reserved box; denotes the area of the first preliminary candidate anchor box; The low-score preliminary anchor box with an intersection-over-union ratio higher than an intersection-over-union ratio threshold (defined based on a task characteristic and a balance requirement of an evaluation index, such as 0.5) is removed, and the process of selecting a reserved box and removing a low-score preliminary anchor box is repeated until all anchor boxes are processed, to generate a set of refined candidate anchor boxes.

[0039] S3.4, a size scaling ratio of the depth feature map relative to the optical surface image is calculated, and the set of refined candidate anchor boxes is forwardly mapped to a coordinate space of the optical surface image based on the size scaling ratio, to generate a set of candidate defect region coordinates.

[0040] It should be noted that the width direction scaling ratio is obtained by calculating a ratio of the optical surface image width to the depth feature map width, and the height direction scaling ratio is obtained by calculating a ratio of the optical surface image height to the depth feature map height, and the expression is, ; ; wherein, the width direction scaling ratio is represented by; the optical surface image width is represented by; the depth feature map width is represented by; the height direction scaling ratio is represented by; the optical surface image height is represented by; the depth feature map height is represented by; Based on the width direction scaling ratio and the height direction scaling ratio, coordinate data corresponding to each refined candidate anchor box in the optical surface image is calculated, including a center point coordinate, a width, and a height value, and coordinate data corresponding to all refined candidate anchor boxes in the optical surface image is combined to generate the set of candidate defect region coordinates; the expression for calculating the coordinate data is, ; ; ; ; wherein, the center point coordinate of the refined candidate anchor box on the optical surface image is represented by; the center point coordinate of the refined candidate anchor box on the optical surface image is represented by; the width of the refined candidate anchor box on the optical surface image is represented by; the height of the refined candidate anchor box on the optical surface image is represented by; the center point coordinate of the refined candidate anchor box on the depth feature map is represented by; the center point coordinate of the refined candidate anchor box on the depth feature map is represented by; the center point coordinate of the refined candidate anchor box on the depth feature map is represented by; the center point coordinate of the refined candidate anchor box on the depth feature map is represented by; a center point of the refined candidate anchor frame on the depth feature map coordinates; a width of the refined candidate anchor frame on the depth feature map a height of the refined candidate anchor frame on the depth feature map

[0041] S4. cropping a local feature map corresponding to each candidate defect region from the depth feature map based on the candidate defect region coordinate set and the depth feature map, and performing scale normalization on the local feature map to generate a candidate region feature map set; S4.1. generating a depth feature map anchor frame set by inversely mapping each anchor frame in the candidate defect region coordinate set to the depth feature map based on the size scaling ratio; It should be noted that based on the width direction scaling ratio and the height direction scaling ratio, the ratio of the center point x coordinate of each anchor frame in the candidate defect region coordinate set to the width direction scaling ratio, the ratio of the center point y coordinate to the height direction scaling ratio, the ratio of the width value to the width direction scaling ratio, and the ratio of the height value to the height direction scaling ratio are calculated to generate anchor frame coordinate data inversely mapped to the depth feature map. The coordinate conversion of the inverse mapping is completed by traversing all the candidate defect region coordinates to generate the depth feature map anchor frame set.

[0042] It should also be noted that the purpose of forward mapping is to convert the detection result on the depth feature map into physical coordinates of the optical surface image that can be understood by the user, to meet the demand for accurate positioning of defect positions in actual application. The coordinate set after forward mapping completely retains the candidate frame information after non-maximum suppression screening. The inverse mapping is to accurately backtrack to the corresponding region of the depth feature map for feature cropping based on the refined physical coordinates, to ensure that the local features used for classification are strictly aligned with the final output defect position, thereby solving the feature misalignment problem caused by non-uniform coordinates.

[0043] S4.2. performing bilinear interpolation feature cropping on each depth feature map anchor frame set on the depth feature map to extract the local feature map of the corresponding region; It should be noted that the center point coordinates and size parameters of each depth feature map anchor box in the depth feature map anchor box set are used to locate the corresponding rectangular region range on the depth feature map, and the feature values of each sampling point in the rectangular region are calculated by bilinear interpolation: according to the continuous coordinates of the sampling point, the nearest four integer coordinate sampling points (lower left, lower right, upper left and upper right) of the sampling point on the depth feature map are located, the horizontal distance difference between the sampling point and the left sampling point and the vertical distance difference between the sampling point and the lower sampling point are calculated, the ratio of the horizontal distance difference to the horizontal spacing of the sampling point is taken as the horizontal weight coefficient, and the ratio of the vertical distance difference to the vertical spacing of the sampling point is taken as the vertical weight coefficient; based on the horizontal weight coefficient, linear interpolation is performed on the four sampling points in the horizontal direction, and based on the vertical weight coefficient, quadratic linear interpolation is performed on the linear interpolation result in the horizontal direction to obtain the feature value of the target sampling point. The expression of bilinear interpolation calculation is, ; wherein, denotes the feature value of the sampling point ; denotes the horizontal weight coefficient; denotes the vertical weight coefficient; denotes the feature value of the lower left corner sampling point ; denotes the feature value of the lower right corner sampling point ; denotes the feature value of the upper left corner sampling point ; denotes the feature value of the upper right corner sampling point ;

[0044] All feature values of the sampling points in the rectangular region are traversed to generate a local feature map matching the size of the depth feature map anchor box; the repeated bilinear interpolation operation of all sampling points in the rectangular region is performed to generate a local feature map set.

[0045] S4.3, normalize the local feature maps of different sizes to a fixed size by bilinear interpolation, and integrate all local feature maps of the fixed size in the same order in the candidate defect region coordinate set to generate a candidate region feature map set.

[0046] It should be noted that the target size (such as 7x7) is defined based on the task requirements and the network structure design, the target size is divided into a target size grid (a uniform two-dimensional coordinate point array), and each target size grid point corresponds to a position coordinate on the target feature map (a local feature map with a uniform fixed size); The local feature maps of different sizes are scaled to the target size grid by bilinear interpolation operation, and the specific operation is to calculate the corresponding continuous coordinate position of each sampling point in the target size grid in the local feature map, and the expression is, ; ; wherein, denotes the corresponding continuous horizontal coordinate mapped into the local feature map; denotes the corresponding continuous vertical coordinate mapped into the local feature map; denotes the actual width of the local feature map; denotes the actual height of the local feature map; denotes the horizontal coordinate of the current sampling point in the target size grid; denotes the vertical coordinate of the current sampling point in the target size grid; denotes the target size width; denotes the target size height; denotes the parameter category of the local feature map; denotes the parameter category of the target size; The feature values of the target sampling points are calculated by distance weighted average calculation according to the feature values of the four nearest sampling points around the continuous coordinate position. After the bilinear interpolation calculation of all grid points in the target size grid is completed, the fixed size local feature map is generated. All fixed size local feature maps are arranged and integrated according to the original order of the anchor box in the candidate defect region coordinate set, to generate a candidate region feature map set with consistent channel number and uniform spatial size.

[0047] S5, each feature map in the candidate region feature map set is compressed into a feature vector, and the feature vector is subjected to semantic information extraction through nonlinear transformation and feature dimension reduction, to generate a candidate region feature vector set; S5.1, each feature map in the candidate region feature map set is compressed into a feature vector through a global average pooling operation, to generate an initial feature vector set; It should be noted that the feature values of all spatial positions in each channel of each candidate region feature map in the candidate region feature map set are summed through the global average pooling operation, and the ratio of the sum result to the total number of spatial positions of the current channel is calculated to obtain the average feature value of each channel. The average feature values of all channels are connected and combined into a one-dimensional vector in the order of the channels. The global average pooling operation is independently performed on each candidate region feature map in the candidate region feature map set, and the one-dimensional vector of each candidate region feature map is combined to generate an initial feature vector set.

[0048] It should also be noted that the channel is an inherent attribute of the deep learning feature extraction network when generating a deep feature map, which represents different feature types and refers to the number of feature values contained in each spatial position of the feature map.

[0049] S5.2, project each initial feature vector in the initial feature vector set to a high-dimensional space and apply a ReLU activation function to generate a high-dimensional feature vector; It should be noted that the high-dimensional weight matrix is defined based on the relationship between the length of the input initial feature vector and the target length of the high-dimensional feature vector (predefined based on task requirements and computational resource limitations), specifically, the number of rows of the high-dimensional weight matrix is set according to the number of elements of the input initial feature vector, the number of columns of the high-dimensional weight matrix is set according to the number of elements of the output high-dimensional feature vector, and a high-dimensional weight matrix with a specific number of rows and columns is constructed; the high-dimensional weight matrix elements are filled with values sampled from a uniform distribution using a random number generator; The product of each element of the initial feature vector and each element in the corresponding row of the high-dimensional weight matrix is calculated, and the sum of all product results is summed by row to obtain each element value of the projection vector, thereby mapping the initial feature vector to a new dimensional space to generate a projection vector; a ReLU activation function is applied to each element of the projection vector to generate a high-dimensional feature vector, and the expression of the ReLU activation function is, ; wherein, denotes the ReLU activation function; denotes the input value, i.e. each element value of the projection vector; each element value of the projection vector is checked one by one, if the current element value is less than zero, the current element value is set to zero, if the current element value is greater than or equal to zero, the current element value is kept unchanged.

[0050] S5.3, project the high-dimensional feature vector to a low-dimensional space to generate a reduced-dimensional feature vector; It should be noted that the reduced-dimensional weight matrix is defined based on the relationship between the length of the input high-dimensional feature vector and the target length of the output reduced-dimensional feature vector, specifically, the number of rows of the reduced-dimensional weight matrix is set according to the number of elements of the input high-dimensional feature vector, the number of columns of the reduced-dimensional weight matrix is set according to the number of elements of the output reduced-dimensional feature vector, and the number of columns is less than the number of rows to achieve the goal of dimension reduction, a reduced-dimensional weight matrix with more rows than columns is constructed, and the reduced-dimensional weight matrix elements are filled with values sampled from a uniform distribution using a random number generator; The product of each element of the high-dimensional feature vector and each element in the corresponding row of the reduced-dimensional weight matrix is calculated, and the sum of all product results is summed by row to obtain each element value of the reduced-dimensional feature vector, since the number of columns of the reduced-dimensional weight matrix is small, the dimension of the high-dimensional feature vector is reduced, and the high-dimensional feature vector is mapped to a low-dimensional space to generate a reduced-dimensional feature vector.

[0051] S5.4, perform batch normalization processing on the reduced-dimensional feature vector to generate a standardized feature vector; It should be noted that the mean and variance of each feature dimension are calculated for each batch of reduced dimension feature vectors, and the expressions for calculating the mean and variance are, ; ; wherein, represents the mean of the th feature dimension; represents the variance of the th feature dimension; represents the number of samples in the batch, i.e., the number of reduced dimension feature vectors in a batch; represents the value of the th sample in the th feature dimension; represents the sample index, whose value range is 1- ; The difference between the mean and the feature value in each reduced dimension feature vector is calculated to obtain the centralized feature value; the sum square root of the variance and a minimum constant (defined based on numerical stability requirements and floating point calculation precision to prevent division by zero error when the variance is zero, such as 10 -5 The product of the normalized feature value and a scaling parameter (defined based on the flexibility requirements of feature distribution) is calculated, and the sum of the product result and an offset parameter (defined based on the data centering recovery requirements) is calculated to generate a standardized feature vector.

[0052] S5.5, sort the standardized feature vectors in the original order of the candidate region feature map set to generate a candidate region feature vector set.

[0053] It should be noted that the standardized feature vectors are arranged in the original order of the corresponding anchor box in the candidate region feature map set, ensuring that each standardized feature vector corresponds to a spatial position in the candidate region feature vector set, and generating a candidate region feature vector set that is completely consistent with the candidate region feature map set in order.

[0054] It should also be noted that the prior art directly compresses the candidate region feature map into a feature vector through a global average pooling operation, which can reduce the computational complexity and preserve the channel semantic information, but lacks the ability to deeply mine the non-linear relationship between features and focus information, resulting in insufficient semantic expression ability for weak defects on optical surfaces; the present scheme enhances the non-linear expression ability of features through high-dimensional projection and ReLU activation, then extracts key semantic information through dimension reduction projection, and finally performs batch normalization to stabilize the feature distribution, generating a standardized feature vector with high discriminability, solving the problem of missing weak defect detection caused by insufficient feature information density.

[0055] S6. Inputting the generated candidate region feature vector set into a deep learning classifier, and outputting a defect category determination result and a confidence score.

[0056] S6.1. Inputting each candidate region feature vector in the candidate region feature vector set into a full connection layer for calculation, and generating an initial score vector of each defect category; It should be noted that the number of rows of the full connection layer weight matrix is determined according to the length of the input candidate region feature vector and the number of output defect categories (defined based on actual physical defect types and detection task requirements, such as the number of categories such as scratches and stains), the number of columns is the feature dimension of the input candidate region feature vector, and each row and column position stores an adjustable value representing the association strength of the feature and the category. All adjustable values are initialized using a random number generator to form a full connection layer weight matrix composed of row and column structures and containing specific values at each position; The product of each element of each candidate region feature vector in the candidate region feature vector set and the corresponding element in each column of the full connection layer weight matrix is calculated, the sum of the product of each column is calculated, and the initial score of the corresponding defect category of the current column is obtained. The initial score vector is generated by combining the initial scores of all defect categories.

[0057] S6.2. Converting the initial score vector into a probability distribution vector through a Softmax function to generate a probability value of each candidate region belonging to each defect category; It should be noted that the initial score of each defect category in the initial score vector is subjected to exponential operation through the Softmax function to obtain an exponential score value, the sum of the exponential score values of all defect categories is calculated, the ratio of the exponential score value of each defect category to the sum of the exponential score values is taken as a normalized probability value, and the normalized probability values of all defect categories are combined in category order to generate a probability distribution vector. The mathematical expression of the Softmax function is, ; Wherein, represents the normalized probability value of the i-th defect category; represents the initial score of the i-th defect category; represents the total number of defect categories; represents the category index, and the value range is 1- ; represents the exponential operation on the initial score of the i-th defect category; represents the summation index, which is a loop variable, and the value range is 1- ; represents the loop summation operation on the i-th defect category; represents the summation index, which is a loop variable, and the value range is 1- ; represents the loop summation operation on the i-th defect category; an initial score of each defect category.

[0058] S6.3, extract the defect category corresponding to the maximum probability value as the defect category determination result, and output the maximum probability value as the confidence score.

[0059] It should be noted that all the defect categories to be identified are clearly defined and form a name list, and a continuous and increasing integer number is assigned to each defect category in sequence from zero according to the name list, forming a static comparison table binding the number and the defect category; The probability values of all defect categories in the probability distribution vector are compared, the maximum probability value and the corresponding defect category index are found out, the defect category index is input into the static comparison table, the corresponding actual defect type name is found out as the defect category determination result, and the maximum probability value is directly output as the confidence score.

[0060] The embodiment also provides a computer device suitable for the optical surface defect data detection method based on deep learning, which comprises a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions to realize the optical surface defect data detection method based on deep learning proposed in the above embodiment.

[0061] The computer device can be a terminal, and the computer device comprises a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device. In addition, the input device can be an external keyboard, touchpad or mouse, etc.

[0062] The embodiment also provides a storage medium on which a computer program is stored, the program being executed by a processor to implement the method for detecting optical surface defect data based on deep learning proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or an optical disk.

[0063] To sum up, the application breaks through the limitation of a single imaging mode by optical scattering physical model driven multi-modal data synthesis, dynamically analyzes the scattering characteristics of defects under multi-physical field coupling, enhances the recognizability of weak defects in a complex scattering environment, and solves the defect missing detection problem; the symmetry interaction of the encoder and the decoder path is established through the bidirectional attention feedback mechanism, the cross-level features are fused through bidirectional weight modulation, the consistency expression of local subtle texture and global structure semantics is coordinated, the positioning redundancy of irregular composite defects and the classification confidence fluctuation are eliminated, and substantial breakthroughs in detection accuracy and robustness are achieved.

[0064] It should be noted that the above embodiments are only used to illustrate the technical solutions of the application rather than limit the application. Although the application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the application, and all of them should be covered in the scope of the claims of the application.

Claims

1. A method for detecting optical surface defect data based on deep learning, characterized in that: include, An optical scattering physics model is constructed, and the collected optical surface images are input into the optical scattering physics model for multimodal data synthesis to generate multimodal image data; A deep learning feature extraction network is constructed, and multimodal image data is input. Multi-scale feature fusion and enhancement are performed through a bidirectional attention feedback mechanism to generate a deep feature map. A deep learning-based region generation method is used to perform spatial domain analysis on the deep feature map, locate the coordinates of potential defect regions, and generate a set of candidate defect region coordinates. The local feature map corresponding to each candidate defect region is cropped from the depth feature map, and the scale of the local feature map is normalized to generate a set of candidate region feature maps. Each feature map in the candidate region feature map set is compressed into a feature vector, and semantic information is extracted from the feature vector through nonlinear transformation and feature dimensionality reduction to generate a candidate region feature vector set. The set of feature vectors of candidate regions is input into a deep learning classifier, which outputs the defect category determination result and confidence score.

2. The method for detecting optical surface defects based on deep learning as described in claim 1, characterized in that: The specific steps for constructing the optical scattering physical model are as follows. A theoretical framework layer is constructed based on the existing bidirectional reflection distribution function theory, a physical property calculation layer is constructed based on the light scattering characteristics of microsurfaces, and a parameter configuration layer is constructed based on the physical property parameter types. An optical scattering physics model is constructed by combining the theoretical framework layer, the physical property calculation layer, and the parameter configuration layer.

3. The method for detecting optical surface defects based on deep learning as described in claim 2, characterized in that: The specific steps for generating multimodal image data are as follows: Optical surface images are acquired using an image sensor and a controllable illumination structure, and the optical surface images are analyzed using an inverse rendering method to extract physical property parameter values. The physical property parameter values ​​are assigned to the parameter configuration layer, the values ​​of the parameter configuration layer are called according to the rules of the theoretical framework layer, Monte Carlo rendering is performed in the physical property calculation layer to generate a composite image under the current lighting conditions; after traversing all preset lighting conditions, multimodal image data is generated.

4. The method for detecting optical surface defects based on deep learning as described in claim 3, characterized in that: The specific steps for constructing the deep learning feature extraction network are as follows: A topology layer is constructed based on an encoder-decoder architecture, a feature transformation layer is constructed based on feature extraction and resolution transformation operations, and a feature fusion layer is constructed based on a bidirectional attention feedback mechanism. A deep learning feature extraction network is constructed based on a topology layer, a feature transformation layer, and a feature fusion layer.

5. The method for detecting optical surface defects based on deep learning as described in claim 4, characterized in that: The specific steps for generating the depth feature map are as follows: Multimodal image data is input into a deep learning feature extraction network, and the multimodal image data is encoded and decoded through a topological structure layer; The feature transformation layer performs feature extraction and resolution transformation, while the feature fusion layer generates a deep feature map through feature enhancement and fusion.

6. The method for detecting optical surface defects based on deep learning as described in claim 5, characterized in that: The specific steps for generating the set of candidate defect region coordinates are as follows: A multi-scale feature pyramid is constructed based on the deep feature map. An anchor box with different size and aspect ratio is preset at each spatial location of the multi-scale feature pyramid to generate an initial set of anchor boxes. Based on the multi-scale feature pyramid, the defect confidence score and bounding box coordinate offset of each initial anchor box are calculated to generate a preliminary set of candidate anchor boxes; The preliminary candidate anchor boxes are sorted according to the defect confidence score, the selected boxes are retained, and the cross-union ratio is calculated based on the retained boxes. The preliminary candidate anchor boxes whose cross-union ratio does not exceed the preset cross-union ratio threshold are extracted to generate a refined candidate anchor box set. Calculate the scaling ratio of the depth feature map relative to the optical surface image, and based on the scaling ratio, forward map the refined candidate anchor box set to the coordinate space of the optical surface image to generate a set of candidate defect region coordinates.

7. The method for detecting optical surface defects based on deep learning as described in claim 6, characterized in that: The specific steps for cropping the local feature map corresponding to each candidate defect region from the depth feature map based on the set of candidate defect region coordinates and the depth feature map are as follows. Based on the size scaling ratio, each anchor box in the candidate defect region coordinate set is inversely mapped to the depth feature map to generate a depth feature map anchor box set. Bilinear interpolation feature clipping is performed on each set of anchor boxes in the depth feature map to extract the local feature map of the corresponding region.

8. The method for detecting optical surface defects based on deep learning as described in claim 7, characterized in that: The steps for scaling the local feature maps to generate a set of candidate region feature maps are as follows: Local feature maps of different sizes are normalized to a fixed size using bilinear interpolation; All fixed-size local feature maps are integrated in the same order as the candidate defect region coordinate set to generate a candidate region feature map set.

9. The method for detecting optical surface defects based on deep learning as described in claim 8, characterized in that: The specific steps for creating the candidate region feature vector set are as follows. The initial feature vector set is generated by compressing each feature map in the candidate region feature map set into an initial feature vector through a global average pooling operation. Each initial feature vector in the initial feature vector set is projected into a high-dimensional space and the ReLU activation function is applied to generate a high-dimensional feature vector. Projecting high-dimensional feature vectors onto a low-dimensional space generates dimensionality-reduced feature vectors. Batch normalization is performed on the dimensionality-reduced feature vectors to generate standardized feature vectors; The standardized feature vectors are sorted according to the original order of the candidate region feature map set to generate a candidate region feature vector set.

10. The method for detecting optical surface defects based on deep learning as described in claim 9, characterized in that: The specific steps for inputting the candidate region feature vector set into the deep learning classifier and outputting the defect category determination result and confidence score are as follows. Each candidate region feature vector in the candidate region feature vector set is input into the fully connected layer for calculation to generate an initial score vector for each defect category; The initial score vector is converted into a probability distribution vector using the Softmax function, generating the probability value of each candidate region belonging to each defect category; Extract the defect category corresponding to the highest probability value as the defect category determination result, and use the highest probability value as the confidence score.

Images