Surface roughness detection method and system based on white light interference image

Abstract

The invention discloses a surface roughness detection method and system based on a white light interference image. The method comprises the following steps: acquiring a training image set of sample surface roughness; pre-training the FAMNet model based on the training image set, extracting feature information of the white light interference image through a feature extraction network, and enhancing and fusing the feature information through a fusion attention module by adopting a channel attention and position attention mechanism; after pre-training, the FAMNet model is further trained, the detection capability of the FAMNet model on different surface roughness is optimized, and the performance of the model can be evaluated through a plurality of evaluation indexes; and finally, inputting a to-be-detected image into the trained FAMNet model to obtain a detection result. According to the method, the lightweight semantic segmentation model FAMNet fused with the attention mechanism is constructed, the recognition capability of the surface roughness of different areas is enhanced, accurate, efficient and rapid roughness detection is carried out on the metal surface, and the method is suitable for real-time monitoring of the surface roughness in industrial production.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a surface roughness detection method and system based on white light interference images. Background Art

[0002] With the continuous development of computer vision technology, the application of deep learning models to detect the surface roughness of metal blocks to achieve intelligent guidance of industrial production has become more and more widespread, and the industry has continuously realized automated and intelligent detection.

[0003] Surface roughness refers to the distance between tiny peaks and valleys on the surface of a workpiece, which reflects the microscopic unevenness of the workpiece surface. The smaller the surface roughness, the smoother the workpiece surface. The roughness of the workpiece surface will affect the properties of the workpiece such as wear resistance, fatigue resistance, vibration strength, contact stiffness and noise. In the manufacturing industry, the service life and reliability of the workpiece depend to a large extent on the surface roughness. The greater the surface roughness, the smaller the effective area in contact with the workpiece surface, the greater the pressure, which leads to greater friction resistance and ultimately wear of the machined surface. Modern mechanical surface processing requires efficient and accurate evaluation of surface roughness.

[0004] With the rapid development of the manufacturing industry, the traditional roughness detection method (stylus method) can no longer meet the requirements of fast and efficient full inspection of roughness during mass production. The stylus method has the problems of slow detection speed, damage to the workpiece surface and low detection accuracy, and cannot achieve high-speed surface roughness measurement. Although non-contact optical microscopes, interferometers, etc. can achieve non-contact, high-precision roughness measurement, they are slow to measure, expensive, and difficult to deploy in complex environments. They are usually used as laboratory offline measurements. The roughness detection method based on machine vision has the characteristics of non-contact, fast detection speed, low price and easy installation, and has achieved certain research results. The different texture information of the sample surface presented by different mechanical processing methods brings difficulties to the detection of roughness. It is important to provide a fast and efficient real-time roughness detection method suitable for a variety of roughness measurements.

[0005] The current online roughness detection method based on machine vision usually adopts a classification method, that is, a roughness value is obtained for an input image. However, the surface roughness value covered in an image is not a uniform value. Therefore, how to achieve more accurate detection of surface roughness has become a technical problem that technicians in this field need to solve urgently.

[0006] Therefore, the present invention proposes a method that can detect different roughness values ​​of an image online through semantic segmentation, so as to make the detection result more accurate. Summary of the invention

[0007] The purpose of the present invention is to provide a surface roughness detection method and system based on white light interference images, to construct a lightweight semantic segmentation model FAMNet integrating an attention mechanism, to detect different roughness values ​​for an image, to improve the recognition ability of surface roughness in different areas, to make the detection results more precise, and to be able to perform accurate, efficient and fast roughness detection on metal surfaces.

[0008] To achieve the above object, the present invention provides the following solutions:

[0009] A method for detecting surface roughness based on white light interference images, the method comprising the following steps:

[0010] S1, obtaining a sample surface roughness data set and preprocessing it to obtain a training image set, wherein the training image set includes a plurality of white light interference images of the sample surface roughness;

[0011] S2, pre-training the FAMNet model based on the training image set, wherein the FAMNet model includes a feature extraction network and a fusion attention module, extracting feature information of the white light interference image through the feature extraction network, and enhancing and fusing the feature information through the fusion attention module using channel attention and position attention mechanisms;

[0012] S3, after pre-training, further training the FAMNet model to optimize its detection ability for different surface roughnesses, and obtain the trained FAMNet model as a surface roughness detection model;

[0013] S4, input the image to be detected into the trained FAMNet model to obtain the detection result.

[0014] Furthermore, in S1, obtaining a sample surface roughness data set and preprocessing it to obtain a training image set specifically includes:

[0015] S101, using a white light interference device in conjunction with an image acquisition device to capture a surface roughness image of the sample under set lighting conditions, to ensure that the image can accurately reflect the microstructure of the sample surface, wherein the surface roughness image is a white light interference image;

[0016] S102, determining the roughness value at the interference fringes of each surface roughness image according to the measurement data of the white light interferometer, and using the roughness value as the roughness label of each surface roughness image to guide the FAMNet model to learn the corresponding relationship between surface roughness and image features;

[0017] S103, preprocessing the collected surface roughness image;

[0018] S104, integrating the preprocessed surface roughness images and their corresponding roughness labels to obtain a training image set.

[0019] Furthermore, in S2, the FAMNet model is pre-trained based on the training image set, specifically including:

[0020] S201, the feature extraction network of the FAMNet model uses convolution modules and jump connections to gradually refine and extract pre-fine-grained features to obtain better local features and obtain feature information of the white light interference image;

[0021] S202, the fusion attention module is introduced into the FAMNet model, which combines the channel attention mechanism and the position attention mechanism to enhance the feature information from the spatial and channel perspectives.

[0022] Furthermore, in S201, the feature extraction network of the FAMNet model extracts the pre-fine-grained features step by step through convolution modules and jump connections to obtain better local features and obtain feature information of the white light interference image, specifically including:

[0023] Feature X represents the roughness feature of the white light interference image. Feature X of each stage is obtained by pre-fine-grained feature extraction. Stagei , each stage represents an extraction stage in the FAMNet model. Each stage contains a series of layers (such as convolutional layers, pooling layers, activation layers, etc.), which together complete a certain processing or feature extraction of the input data. The calculation formula is shown in formulas (1)-(8):

[0024] X Stage0 =f(conv 7×7 (x)) (1)

[0025] f(X)=Relu(BN(X)) (2)

[0026] f1=f(conv 1×1 (X)) (3)

[0027] f2=f(conv 3×3 (f1)) (4)

[0028] F main =f(conv 1×1 (d2)) (5)

[0029] block1(X)=F main +f(conv 1×1 (X)) (6)

[0030] block2(X)=F main+Relu(X) (7)

[0031] X Stagei =f(block2(block1(X Stagei-1 )))(i=1,2,3,4) (8)

[0032] Among them, the original size of feature X is H×W×C, where H represents the original height of the feature map, W represents the original width of the feature map, and C represents the original number of channels of the feature map; the FAMNet model feature extraction has 0-5 extraction stages, X Stagei represents the output of the i-th extraction stage, X Stage0 represents the primary features obtained by the 7*7 convolution kernel, and Stagei (i=1,2,3,4) is the result obtained by passing the output of the previous stage Stagei-1 through two processing modules block1 and block2 in sequence, and then processing it through the function f(·); f(·) is the general paradigm of feature extraction; f(X) represents the new extracted features obtained by feature X after the BN layer and the Relu activation function; conv k×k Indicates that a convolution kernel of size k*k is used; features f1 and f2 are extracted from the input feature X through continuous convolution operations, representing the features of the image at different levels respectively; in local feature extraction, f2 is first further processed through a 1×1 convolution layer to obtain the main feature F main , then F main The result of another 1×1 convolutional layer is added to the original input feature X to form the output block1(X) of the first layer block; then, block1(X) is added to the feature X processed by the ReLU activation function to obtain the output block2(X) of the second layer block; finally, the output block1(X) of the first layer block is processed by the second layer block. Stagei-1 ), and get the final output X Stagei .

[0033] Furthermore, in S202, a fusion attention module is introduced into the FAMNet model to combine the channel attention mechanism and the position attention mechanism to enhance feature information from the spatial and channel perspectives, specifically including:

[0034] The fusion attention module first extracts channel-dimensional features based on the channel attention mechanism for the input feature information to capture channel dependency and enhance the connection between different channels, and then extracts spatial-dimensional features based on the position attention mechanism to capture position dependency.

[0035] Furthermore, in S202, a fusion attention module is introduced into the FAMNet model to combine the channel attention mechanism and the position attention mechanism to enhance feature information from the perspective of space and channel. The specific calculation formula is as follows:

[0036] Q1,K1,V1=Conv(X input1 ) (9)

[0037] Q′1,K′1,V′1=Reshape(Q1,K1,V1) (10)

[0038]

[0039] X input2 =X output1 (13)

[0040] Q2,K2,V2=Conv(X input2 ) (14)

[0041] Q′2,K′2,V′2=Reshape(Linear(Q2,K2,V2)) (15)

[0042]

[0043] The FAM module extracts channel-dimensional features from the input features in the first stage to capture channel dependencies and enhance the connection between different channels. In the second stage, it extracts spatial-dimensional features to capture position dependencies and reduce the loss of position information.

[0044] In the first stage, as shown in formula (9), the input feature X is convolved to generate query Q1, key K1 and value V1 vectors of size C×H×W; then, the query Q1, key K1 and value V1 vectors are reshaped into shapes suitable for matrix multiplication Q'1, K'1, V'1, of size C×N, as shown in formula (10);

[0045] Then, the attention weight is obtained by the attention calculation formula, as shown in formula (11), and this weight is then normalized by the Softmax function to obtain the normalized weight;

[0046] Next, the normalized weights are matrix multiplied with the value vector to obtain the weighted value, which is added to the original input feature map X to obtain the output feature map X output1 , as shown in formula (12);

[0047] In the second stage, the output feature map X output1As the new input feature map, the second set of query Q2, key K2 and value V2 vectors are generated through formulas (13)-(14), and adjusted through the linear layer Linear, as shown in formula (15). The new attention weights are calculated again through matrix multiplication and Softmax function, and matrix multiplication is performed with the weighted values. Finally, it is added to the input feature map to obtain the final output feature map X output2 , as shown in formulas (16)-(17).

[0048] Furthermore, in S3, after pre-training, the FAMNet model is further trained, specifically including:

[0049] The FAMNet model was trained on two RTX 2080Ti graphics processors based on the deep learning framework PyTorch, with 20 iterations and a batch size of 64; the Adam optimizer was used to optimize the model parameters with a weight decay of 0.0001 to improve the FAMNet model's ability to learn and generalize surface roughness features.

[0050] Furthermore, S3 also includes model performance evaluation:

[0051] The cross entropy loss was selected as the loss function, and accuracy, precision, recall and specificity were used as the evaluation indicators of the model to evaluate and optimize the trained FAMNet model.

[0052] The present invention also provides a system for detecting surface roughness based on white light interference images, which is used to perform the above-mentioned method for detecting surface roughness based on white light interference images, comprising:

[0053] An image processing module is used to obtain a sample surface roughness data set and perform preprocessing to obtain a training image set, wherein the training image set includes a plurality of white light interference images of the sample surface roughness;

[0054] A model building and training module is used to pre-train the FAMNet model based on a training image set, extract feature information of the white light interference image through a feature extraction network, and enhance and fuse the feature information using a channel attention and position attention mechanism through a fusion attention module, wherein the FAMNet model includes a feature extraction network and a fusion attention module;

[0055] It is also used to further train the FAMNet model after pre-training to optimize its detection ability for different surface roughnesses, and obtain the trained FAMNet model as a surface roughness detection model;

[0056] The detection module is used to input the image to be detected into the trained FAMNet model to obtain the detection result.

[0057] In addition, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the surface roughness detection method based on white light interference images as described above is implemented.

[0058] According to the specific embodiments provided by the present invention, the surface roughness detection method and system based on white light interference image provided by the present invention disclose the following technical effects:

[0059] (1) Construct and train a semantic segmentation FAMNet model based on the fusion attention mechanism. This model optimizes the extraction and recognition capabilities of surface roughness features in white light interference images through the fusion attention mechanism, reduces the computational complexity of the model, and maintains the lightweight characteristics of the FAMNet model.

[0060] (2) The FAMNet model includes a feature extraction network and a fusion attention module. The fusion attention module combines the channel attention mechanism and the position attention mechanism to enhance feature information from the spatial and channel perspectives, thereby improving the model’s ability to learn different surface roughness features.

[0061] (3) In the pre-training stage, the FAMNet model learns the correlation of image features through the self-attention mechanism, effectively integrating the feature information of different regions and improving the model's detection accuracy of surface roughness;

[0062] (4) The cross entropy loss is used as the loss function, and the model is optimized through evaluation indicators such as accuracy, precision, recall and specificity to ensure the high prediction accuracy of the model;

[0063] (5) The white light interference image to be detected is input into the trained FAMNet model to predict the surface roughness, which solves the problem of low efficiency of traditional metal surface roughness detection methods. By integrating the attention mechanism, the performance of surface roughness prediction is improved, which is suitable for real-time monitoring of surface roughness in industrial production. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0065] Figure 1 is a flow chart of a surface roughness detection method based on white light interference image in an embodiment of the present invention;

[0066] Figure 2 A structural diagram of a roughness detection network FAMNet model based on white light interference images provided in an embodiment of the present invention, wherein a is a flowchart of roughness detection of the FAMNet model, and b is a specific framework diagram of FAMNet;

[0067] Figure 3 A network structure diagram of the core FAM module in FAMNet provided in an embodiment of the present invention, wherein a represents the process of extracting features in the channel dimension, and b represents the process of extracting features in the spatial dimension;

[0068] Figure 4 The overall network structure diagram of the FAM module provided by the embodiment of the present invention at different network stages. DETAILED DESCRIPTION

[0069] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0070] The purpose of the present invention is to provide a surface roughness detection method and system based on white light interference images, construct and train a semantic segmentation FAMNet model based on a fusion attention mechanism, input the image to be detected into the trained FAMNet model to obtain the detection result, so as to solve the problem of low efficiency of existing metal surface roughness detection methods.

[0071] The FAMNet model effectively extracts and fuses surface roughness features from white light interference images through its fusion attention mechanism, which not only reduces the size of the model but also improves detection speed and accuracy. The model can provide more accurate surface roughness predictions, thus overcoming the shortcomings of traditional detection methods in terms of speed, ease of deployment, feature extraction capabilities, and detection accuracy.

[0072] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0073] The embodiment of the present invention provides a surface roughness detection method based on feature optimization, and constructs a general roughness detection model suitable for a variety of metal surfaces. In this embodiment, the model is applied to the roughness detection of a variety of metal surfaces. The surface roughness image of the sample is taken at the same position by using an image acquisition device, and then the roughness value corresponding to the sample image is measured by a white light interferometer. Then, the constructed data set is divided into a training set, a validation set and a test set, wherein the training set and the validation set are used for model training, and the test set is used to evaluate the performance of the model. The model construction and optimization process is as follows: First, the FAMNet (Fusion Attention Mechanism Network) model is designed, then the appropriate model parameters are selected, and then the model is trained using the prepared data set. After the training is completed, the model performance is evaluated by the validation set, and the model parameters are adjusted according to the evaluation results to further optimize the model. Finally, the model is evaluated using the test data set to ensure that the model performs as expected. Finally, the model is deployed in an actual surface roughness detection scenario.

[0074] Specifically, Figure 1 As shown, the surface roughness detection method based on white light interference image provided by the embodiment of the present invention includes the following steps:

[0075] S1, obtaining a sample surface roughness data set and preprocessing it to obtain a training image set, wherein the training image set includes a plurality of white light interference images of the sample surface roughness;

[0076] S2, based on the training image set, pre-training the FAMNet model, extracting feature information of the white light interference image through a feature extraction network, and enhancing and fusing the feature information through a fusion attention module using a channel attention and position attention mechanism, wherein the FAMNet model includes a feature extraction network and a fusion attention module;

[0077] S3, after pre-training, further training the FAMNet model to optimize its detection ability for different surface roughnesses, and obtain the trained FAMNet model as a surface roughness detection model;

[0078] This also includes selecting cross entropy loss as the loss function, using accuracy, precision, recall and specificity as the evaluation indicators of the model, and evaluating and optimizing the trained FAMNet model;

[0079] S4, input the image to be detected into the trained FAMNet model to obtain the detection result.

[0080] The FAMNet model constructed and trained by the present invention is as follows Figure 2As shown in the figure, the image to be detected is input into the trained FAMNet model, and the prediction result is obtained through the softmax layer. The FAMNet model consists of multiple stages, each of which contains multiple residual blocks. These residual blocks extract features through convolution operations and further optimize the feature representation through the fusion attention module. The fusion attention module combines channel attention and position attention to improve the model's recognition accuracy for different surface roughness features. In this way, the FAMNet model can effectively process white light interference images and accurately predict surface roughness, solving the limitations of traditional detection methods in speed, deployment convenience, feature extraction capabilities, and detection accuracy.

[0081] For example, in S1, obtaining a sample surface roughness dataset and preprocessing it to obtain a training image set specifically includes:

[0082] S101, using a white light interference device in conjunction with an image acquisition device to capture a surface roughness image of the sample under set lighting conditions, to ensure that the image can accurately reflect the microstructure of the sample surface, wherein the surface roughness image is a white light interference image;

[0083] S102, determining the roughness value at the interference fringes of each surface roughness image according to the measurement data of the white light interferometer, and using the roughness value as the roughness label of each surface roughness image to guide the FAMNet model to learn the corresponding relationship between surface roughness and image features;

[0084] S103, preprocessing the collected surface roughness image, including denoising, contrast enhancement, size standardization, etc., to improve image quality and ensure consistency and effectiveness of model training;

[0085] S104, integrating the preprocessed surface roughness images and their corresponding roughness labels to form a final training image set, providing a data basis for subsequent model training.

[0086] Specifically, in this embodiment, first, an image acquisition device is used to capture the surface roughness image of the sample at the same position. Then, a white light interferometer is used to obtain the roughness value corresponding to the sample image as a label value. Finally, the constructed surface roughness image data set is divided into a training set, a validation set, and a test set, wherein the training set and the validation set are used to train the model, and the test set is used to evaluate the performance of the model. The data set contains a total of 6093 images. The data set is divided into a training set, a validation set, and a test set in a ratio of 7:2:1.

[0087] For example, in S2, the FAMNet model is pre-trained based on the training image set, specifically including:

[0088] S201, the feature extraction network of the FAMNet model uses convolution modules and jump connections to gradually refine and extract pre-fine-grained features to obtain better local features and obtain feature information of the white light interference image;

[0089] S202, the fusion attention module is introduced into the FAMNet model, which combines the channel attention mechanism and the position attention mechanism to enhance the feature information from the spatial and channel perspectives;

[0090] The fused attention module can improve the model's ability to recognize and segment surface roughness features in white light interference images while maintaining computational efficiency.

[0091] The FAMNet model calculates the correlation of each channel in the white light interference image through the fusion attention mechanism, which is used as the information of the interaction between channels. The FAMNet model fuses the feature information of different channels to obtain the mixed channel feature information, thereby integrating the features from the channel perspective. This process involves inputting the feature information of different channels into two paths respectively: one is the channel self-attention calculation path, which is used to evaluate the correlation between channels; the other is the feature fusion path, which is used to merge the feature information of different channels. In this way, each channel is fused with the features of other channels to obtain enhanced channel features, thereby improving the model's ability to recognize surface roughness features.

[0092] This embodiment uses a feature extraction network to extract feature information of images for pre-training parameter learning based on images selected from a training image set in the pre-training stage, and interacts and fuses the feature information of the images through a convolution module and a fusion attention module.

[0093] In this embodiment, the FAMNet model directly receives the white light interference image as input, and extracts and optimizes the features of surface roughness through its internal fusion attention mechanism (FAM). The model effectively identifies and distinguishes different roughness levels in the image by combining channel attention and position attention. During the training phase, FAMNet enhances the representation ability of features by fusion attention modules, and predicts through the softmax layer, thereby achieving accurate detection of surface roughness. After pre-training, the FAMNet model can be used for online prediction, which simplifies the model structure, reduces the number of parameters, improves the detection speed, and facilitates deployment in actual industrial environments. The lightweight design of the FAMNet model makes it easier to integrate into existing manufacturing processes while maintaining high detection accuracy.

[0094] Furthermore, in S201, the feature extraction network of the FAMNet model extracts the pre-fine-grained features step by step through convolution modules and jump connections to obtain better local features and obtain feature information of the white light interference image, specifically including:

[0095] In this embodiment, the fusion attention module is used to extract the feature information of the white light interference image and perform feature fusion; in order to improve the efficiency of feature extraction and capture more refined local features, the FAMNet model uses large convolution kernels to expand the receptive field, thereby capturing a wider range of contextual information; in addition, by introducing skip connections and layer-wise refinement strategies, the model further enhances the expressiveness of features and ensures that high-quality local features can be obtained at different scales; these enhanced features are then input into the multi-layer perceptron to further extract and fuse feature information, and finally obtain optimized features for surface roughness detection. This process improves the model's ability to recognize different roughness features in white light interference images.

[0096] The network structure of FAMNet is shown in the figure Figure 2 As shown in Figure 1, better local features are extracted by extracting pre-fine-grained features, which is beneficial to improving the accuracy and performance of the final recognition. Feature X represents the roughness feature of the white light interference image. The feature X of each extraction stage is obtained by pre-fine-grained feature extraction. Sagei , each stage represents an extraction stage in the FAMNet model. Each stage contains a series of layers (such as convolutional layers, pooling layers, activation layers, etc.), which together complete a certain processing or feature extraction of the input data. The calculation formula is shown in formulas (1)-(8):

[0097] X Stage0 =f(conv 7×7 (x)) (1)

[0098] f(X)=Relu(BN(X)) (2)

[0099] f1=f(conv 1×1 (X)) (3)

[0100] f2=f(conv 3×3 (f1)) (4)

[0101] F main =f(conv 1×1 (d2)) (5)

[0102] block1(X)=F main +f(conv 1×1 (X)) (6)

[0103] block2(X)=F main +Relu(X) (7)

[0104] X Stagei =f(block2(block1(X Stagei-1 )))(i=1,2,3,4) (8)

[0105] Among them, the original size of feature X is H×W×C, where H represents the original height of the feature map, W represents the original width of the feature map, and C represents the original number of channels of the feature map; the FAMNet model feature extraction has 0-5 extraction stages, X Stagei represents the output of the i-th extraction stage, X Stage0 represents the primary features obtained by the 7*7 convolution kernel, and Stagei (i=1,2,3,4) is the result obtained by passing the output of the previous stage Stagei-1 through two processing modules block1 and block2 in sequence, and then processing it through the function f(·); f(·) is the general paradigm of feature extraction; f(X) represents the new extracted features obtained by feature X after the BN layer and the Relu activation function; conv k×k Indicates that a convolution kernel of size k*k is used; features f1 and f2 are extracted from the input feature X through continuous convolution operations, representing the features of the image at different levels respectively; in local feature extraction, f2 is first further processed through a 1×1 convolution layer to obtain the main feature F main , then F main The result of another 1×1 convolutional layer is added to the original input feature X to form the output block1(X) of the first layer block; then, block1(X) is added to the feature X processed by the ReLU activation function to obtain the output block2(X) of the second layer block; finally, the output block1(X) of the first layer block is processed by the second layer block. Stagei-1 ), and get the final output X Stagei This output will be used for subsequent surface roughness detection tasks. This process shows how features are extracted step by step, providing detailed feature information for accurate detection of surface roughness.

[0106] Furthermore, in S202, a fusion attention module is introduced into the FAMNet model to combine the channel attention mechanism and the position attention mechanism to enhance feature information from the spatial and channel perspectives, specifically including:

[0107] The network structure of the fusion attention module (FAM) is as follows Figure 3 As shown in the figure, the fusion attention module first extracts channel dimension features based on the channel attention mechanism for the input feature information to capture channel dependency and enhance the connection between different channels, and then extracts spatial dimension features based on the position attention mechanism to capture position dependency and reduce the loss of position information.

[0108] In order to improve the efficiency of feature extraction, the fusion attention module combines channel attention and position attention to generate an attention weight mask to adjust the size of feature weights. This fusion attention mechanism provides effective feature fusion for the pre-training stage and enhances the model's ability to recognize different surface roughness features.

[0109] The fusion attention module enhances the network model's ability to perceive location information and learn local features. The specific calculation formulas (9)-(17) are as follows:

[0110] Q1,K1,V1=Conv(X input1 ) (9)

[0111] Q′1,K′1,V′1=Reshape(Q1,K1,V1) (10)

[0112]

[0113] X input2 =X output1 (13)

[0114] Q2,K2,V2=Conv(X input2 ) (14)

[0115] Q′2,K′2,V′2=Reshape(Linear(Q2,K2,V2)) (15)

[0116]

[0117] The FAM module extracts channel-dimensional features from the input features in the first stage to capture channel dependencies and enhance the connection between different channels. In the second stage, it extracts spatial-dimensional features to capture position dependencies and reduce the loss of position information.

[0118] In the first stage, as shown in formula (9), the input feature X generates query Q1, key K1 and value V1 vectors with a size of C×H×W through convolution operation; then, the query Q1, key K1 and value V1 vectors are reshaped into a shape Q'1, K'1, V'1 suitable for matrix multiplication, with a size of C×N, as shown in formula (10);

[0119] Then, the attention weight is obtained by the attention calculation formula, as shown in formula (11), and this weight is then normalized by the Softmax function to obtain the normalized weight;

[0120] Next, the normalized weights are matrix multiplied with the value vector to obtain the weighted value, which is added to the original input feature map X to obtain the output feature map X output1 , as shown in formula (12);

[0121] In the second stage, the output feature map X output1 As the new input feature map, the second set of query Q2, key K2 and value V2 vectors are generated through formulas (13)-(14), and adjusted through the linear layer Linear, as shown in formula (15). The new attention weights are calculated again through matrix multiplication and Softmax function, and matrix multiplication is performed with the weighted values. Finally, it is added to the input feature map to obtain the final output feature map X output2 , as shown in formulas (16)-(17).

[0122] The design of the entire fusion attention module allows the model to weight and fuse features at different stages, thereby enhancing the model's ability to recognize surface roughness features. Figure 3 The detailed structural design of the fused attention mechanism is presented, which includes operations such as feature transfer, reshaping, and transposition, which together achieve effective feature fusion and enhancement.

[0123] Furthermore, in S3, after pre-training, the FAMNet model is further trained, specifically including:

[0124] The FAMNet model was trained on two RTX 2080Ti graphics processors based on the deep learning framework PyTorch, with 20 iterations and a batch size of 64; the Adam optimizer was used to optimize the model parameters with a weight decay of 0.0001 to improve the FAMNet model's ability to learn and generalize surface roughness features.

[0125] In addition, the embodiment of the present invention also uses the test set divided from the sample surface roughness data set in S1 to predict the roughness of the trained FAMNet model. The cross entropy loss is selected as the loss function, and the mean intersection-over-union ratio (MioU), pixel accuracy (PA) and mean accuracy (MA) are used as evaluation indicators of the model, specifically including:

[0126] In this embodiment, the loss function calculation formula is shown in formula (18), where N represents the total number of images in the data set, H×W represents the resolution size of the image, and Y ij Defined as the label of image i of category j, M ij It is the probability output of category j obtained by inputting picture i into the model.

[0127]

[0128] MIoU is used as the main evaluation indicator. MIoU is an indicator widely used in the evaluation of semantic segmentation algorithms. It can comprehensively consider the prediction accuracy of each category and provide an overall evaluation result. By calculating MIoU, the performance of different models or different parameter configurations on the semantic segmentation task can be compared, thereby verifying the effectiveness and accuracy of the FAMNet model. IoU (intersection-union ratio) measures the similarity between the prediction result and the true label by calculating the ratio of the overlapping area to its union. Specifically, for each category, the IoU of the category can be calculated and the average value can be taken to obtain MIoU. The calculation formula is shown in formula (19).

[0129]

[0130]

[0131] Where w represents the total number of categories. When calculating the final average, the default background category must also be included; p uv Indicates the number of times that category u is incorrectly predicted as category v; p vu represents the number of results whose original category is u but predicted to be category v; p uu Indicates that category u is correctly predicted as category u.

[0132] Finally, the trained FAMNet detection model is used to test and analyze the detection effect.

[0133] Table 1 Prediction accuracy of different models

[0134]

[0135] In this embodiment, compared with the existing models, the parameters and computational complexity of the FAMNet model are minimized. As shown in Table 1, in the machine vision semantic segmentation task, the segmentation recognition accuracy MIoU of the FAMNet model reaches 86.3%, which is higher than the current best-performing model Mask2Former in performance, with a computational performance improvement of about 77.66% and parameters about 63.59% lower than it. The FAMNet model is slightly better than the Mask2Former model. In terms of pixel accuracy, the FAMNet model is 3% higher than UNet++. In terms of average accuracy, the FAMNet model is equivalent to UNet++. The present invention uses parameters and computational complexity as the main evaluation indicators. When the overall segmentation quality evaluation criterion MIoU is comparable, overall, the FAMNet model has achieved the best results so far.

[0136] In actual scene detection, some samples predicted by the present invention are made of oxygen-free copper, and the predicted field size is 1920×1200 pixels. The inference time of the FAMNet model and other model algorithms in real-time detection scenes is shown in Table 2. The model FAMNet constructed by the present invention can infer 30.3 frames per second, which has the best performance compared with other algorithm models. The inference delay time is only 33 milliseconds, which is 69% faster than Mask2former and 18% faster than Unet++. This shows that the method described in the present invention effectively balances the performance of recognition accuracy and speed, and ensures stable and efficient operation with high real-time requirements in the detection environment of the high-precision processing industry.

[0137] Table 2 Reasoning speed of different models

[0138]

[0139] In order to explore the performance effect of the network structure design, experiments were conducted using different backbones. The detailed results are shown in Table 3. ResNet50 and ResNet101 were experimented on the backbone network, and the results reached 77.5 and 84.4 respectively. In terms of segmentation and recognition accuracy, the performance of ResNet101 as the backbone network is about 9% higher than that of ResNet50, but more parameter calculations are required.

[0140] Table 3 Performance comparison of different backbones

[0141]

[0142] In order to study the performance impact of putting the fusion attention mechanism designed by the present invention into the model architecture, the FAM module is placed Figure 4 The different positions shown. The present invention was experimentally tested after three extraction stages (Stage 2, Stage 3, Stage 4). The detailed results are shown in Table 4. It can be seen that when fused after Stage 2, MIoU only reached 78.5, and the effects of PixelAccuracy and MeanAccuracy were also very poor. After being put into Stage 4 for fusion, the performance was improved by about 8%, and after being integrated into Stage 3, it reached 86.3, an increase of about 10%. As shown in Table 4, it can be clearly seen that adding the FAM module after the third extraction stage (Stage 4) can achieve the best roughness recognition accuracy, and the parameter performance and computational complexity are comparable.

[0143] Table 4 Performance comparison of fusion attention mechanism modules placed at different positions

[0144]

[0145] The present invention also provides a system for detecting surface roughness based on white light interference images, which is used to perform the above-mentioned method for detecting surface roughness based on white light interference images, comprising:

[0146] An image processing module is used to obtain a sample surface roughness data set and perform preprocessing to obtain a training image set, wherein the training image set includes a plurality of white light interference images of the sample surface roughness;

[0147] A model building and training module is used to pre-train the FAMNet model based on a training image set, extract feature information of the white light interference image through a feature extraction network, and enhance and fuse the feature information using a channel attention and position attention mechanism through a fusion attention module, wherein the FAMNet model includes a feature extraction network and a fusion attention module;

[0148] It is also used to further train the FAMNet model after pre-training to optimize its detection ability for different surface roughnesses, and obtain the trained FAMNet model as a surface roughness detection model;

[0149] The detection module is used to input the image to be detected into the trained FAMNet model to obtain the detection result.

[0150] In addition, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the surface roughness detection method based on white light interference images as described above is implemented.

[0151] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium, and the program can include the processes of the above-mentioned method embodiments when executed. The storage medium can be a memory, a disk, an optical disk, etc.

[0152] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting surface roughness based on white light interference images, characterized in that: The following steps are involved: S1, acquiring a sample surface roughness data set and preprocessing it to obtain a training image set, wherein the training image set includes a plurality of white light interference images of the sample surface roughness; S2, pre-training the FAMNet model based on the training image set, wherein the FAMNet model includes a feature extraction network and a fusion attention module, extracting feature information of the white light interference image through the feature extraction network, and enhancing and fusing the feature information through the fusion attention module using channel attention and position attention mechanisms; S3, after pre-training, further training the FAMNet model to optimize its detection ability for different surface roughnesses, and obtain the trained FAMNet model as a surface roughness detection model; S4, input the image to be detected into the trained FAMNet model to obtain the detection result.

2. The method for surface roughness detection based on white light interference image according to claim 1, characterized in that: In S1, a sample surface roughness data set is obtained and preprocessed to obtain a training image set, specifically including: S101, using a white light interference device in conjunction with an image acquisition device to capture a surface roughness image of the sample under set lighting conditions, to ensure that the image can accurately reflect the microstructure of the sample surface, wherein the surface roughness image is a white light interference image; S102, determining the roughness value at the interference fringes of each surface roughness image according to the measurement data of the white light interferometer, and using the roughness value as the roughness label of each surface roughness image to guide the FAMNet model to learn the corresponding relationship between surface roughness and image features; S103, preprocessing the collected surface roughness image; S104, integrating the preprocessed surface roughness images and their corresponding roughness labels to obtain a training image set.

3. The method for surface roughness detection based on white light interference image according to claim 1, characterized in that: In S2, the FAMNet model is pre-trained based on the training image set, specifically including: S201, the feature extraction network of the FAMNet model uses convolution modules and jump connections to gradually refine and extract pre-fine-grained features to obtain better local features and obtain feature information of the white light interference image; S202, the fusion attention module is introduced into the FAMNet model, which combines the channel attention mechanism and the position attention mechanism to enhance the feature information from the spatial and channel perspectives.

4. The method for surface roughness detection based on white light interference image according to claim 3, characterized in that: In S201, the feature extraction network of the FAMNet model extracts pre-fine-grained features step by step through convolution modules and jump connections to obtain better local features and obtain feature information of the white light interference image, specifically including: Feature X represents the roughness feature of the white light interference image. The feature X of each extraction stage is obtained by pre-fine-grained feature extraction. Stagei , the calculation formula is shown in formula (1)-(8): X Stage0 =f(conv 7×7 (X)) (1) f(X)=Relu(BN(X)) (2) f1=f(conv 1×1 (X)) (3) f2=f(conv 3×3 (f1)) (4) F main =f(conv 1×1 (f2)) (5) block1(X)=F main +f(conv 1×1 (X)) (6) block2(X)=F main +Relu(X) (7) X Stagei =f(block2(block1(X Stagei-1 ))) (i=1,2,3,4) (8) Among them, the original size of feature X is H×W×C, H represents the original height of the feature map, W represents the original width of the feature map, and C represents the original number of channels of the feature map; the FAMNet model feature extraction has 0-5 extraction stages, X Stagei represents the output of the i-th extraction stage, X Stage0 represents the primary features obtained by the 7*7 convolution kernel; f(·) is the general paradigm of feature extraction; f(X) represents the new extracted features obtained by feature X after the BN layer and the Relu activation function; conv k×k Indicates that a convolution kernel of size k*k is used; features f1 and f2 are extracted from the input feature X through continuous convolution operations, representing the features of the image at different levels respectively; in local feature extraction, f2 is first further processed through a 1×1 convolution layer to obtain the main feature F main , then F main The result of another 1×1 convolutional layer is added to the original input feature X to form the output block1(X) of the first layer block; then, block1(X) is added to the feature X processed by the ReLU activation function to obtain the output block2(X) of the second layer block; finally, the output block1(X) of the first layer block is processed by the second layer block. Stagei-1 ), and get the final output X Stagei .

5. The method for surface roughness detection based on white light interference image according to claim 3, characterized in that: In S202, a fusion attention module is introduced into the FAMNet model to combine the channel attention mechanism and the position attention mechanism to enhance feature information from the perspective of space and channel, specifically including: The fusion attention module first extracts channel-dimensional features based on the channel attention mechanism for the input feature information to capture channel dependency and enhance the connection between different channels, and then extracts spatial-dimensional features based on the position attention mechanism to capture position dependency.

6. The method for detecting surface roughness based on white light interference image according to claim 5, characterized in that: In S202, a fusion attention module is introduced into the FAMNet model to combine the channel attention mechanism and the position attention mechanism to enhance feature information from the perspective of space and channel. The specific calculation formula is as follows: Q 1, K 1, V1=Conv(X input1 ) (9) Q′1,K′1,V′1=Reshape(Q1,K1,V1) (10) X input2 =X output1 (13) Q 2, K 2, V2=Conv(X input2 ) (14) Q′2,K′2,V′2=Reshape(Linear(Q2,K2,V2)) (15) The fusion attention module extracts channel-dimensional features from the input features in the first stage to capture channel dependencies and enhance the connections between different channels, and extracts spatial-dimensional features in the second stage to capture position dependencies. In the first stage, as shown in formula (9), the input feature X is convolved to generate query Q1, key K1 and value V1 vectors of size C × H × W; Next, the query Q1, key K1, and value V1 vectors are reshaped into a shape suitable for matrix multiplication Q'1, K'1, V'1, with a size of C × N, as shown in formula (10); Then, the attention weight is obtained by the attention calculation formula, as shown in formula (11), and this weight is then normalized by the Softmax function to obtain the normalized weight; Next, the normalized weights are matrix multiplied with the value vector to obtain the weighted value, which is added to the original input feature map X to obtain the output feature map X output1 , as shown in formula (12); In the second stage, the output feature map X output1 As the new input feature map, the second set of query Q2, key K2 and value V2 vectors are generated through formulas (13)-(14), and adjusted through the linear layer Linear, as shown in formula (15). The new attention weights are calculated again through matrix multiplication and Softmax function, and matrix multiplication is performed with the weighted values. Finally, it is added to the input feature map to obtain the final output feature map X output2 , as shown in formulas (16)-(17).

7. The surface roughness detection method based on white light interference image according to claim 1, characterized in that: In S3, after pre-training, the FAMNet model is further trained, specifically including: The FAMNet model was trained on two RTX 2080Ti graphics processors based on the deep learning framework PyTorch, with 20 iterations and a batch size of 64; the Adam optimizer was used to optimize the model parameters with a weight decay of 0.0001 to improve the FAMNet model's ability to learn and generalize surface roughness features.

8. The surface roughness detection method based on white light interference image according to claim 1, characterized in that: The S3 also includes model performance evaluation: The cross entropy loss was selected as the loss function, and accuracy, precision, recall and specificity were used as the evaluation indicators of the model to evaluate and optimize the trained FAMNet model.

9. A system for detecting surface roughness based on white light interference images, used for executing the method for detecting surface roughness based on white light interference images as claimed in any one of claims 1 to 8, characterized in that: include: An image processing module is used to obtain a sample surface roughness data set and perform preprocessing to obtain a training image set, wherein the training image set includes a plurality of white light interference images of the sample surface roughness; A model building and training module is used to pre-train the FAMNet model based on a training image set, extract feature information of the white light interference image through a feature extraction network, and enhance and fuse the feature information using a channel attention and position attention mechanism through a fusion attention module, wherein the FAMNet model includes a feature extraction network and a fusion attention module; It is also used to further train the FAMNet model after pre-training to optimize its detection ability for different surface roughnesses, and obtain the trained FAMNet model as a surface roughness detection model; The detection module is used to input the image to be detected into the trained FAMNet model to obtain the detection result.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the surface roughness detection method based on white light interference image as described in any one of claims 1 to 8 is implemented.

Images