Roughness measurement method and device based on multispectral laser speckle image

Abstract

The invention provides a surface roughness measuring method and device based on a multispectral laser speckle image, and belongs to the field of surface quality visual inspection. Comprising the following steps: respectively acquiring speckle images formed by irradiating the surface of a workpiece with lasers of N wavelengths at different angles, and constructing a multispectral laser speckle data set; the speckle images of N wavelengths at a single angle are spliced according to channels and subjected to feature fusion, and then a multispectral laser speckle fusion image is generated; the fused images of the multiple angles form an image sequence, and single images in the sequence are subjected to multi-scale feature extraction and fusion to generate single-angle features; the multi-angle features generate global illumination information through a multi-angle feature aggregation module, and finally a roughness value is calculated through a multi-layer perceptron. Multi-spectral laser speckle coding surface features are adopted, multi-angle information is fused through the multi-angle feature aggregation neural network, and the relation between the light source wavelength, the incident angle and the surface three-dimensional shape and roughness is comprehensively considered, so that high-precision measurement is achieved.

Description

Technical Field

[0001] The present invention relates to a roughness measurement method and device based on multi - spectral laser speckle images, belonging to the fields of optical measurement, deep learning, and computer vision, and is particularly applicable to the quality detection of metal surfaces. Background Art

[0002] Surface roughness is an important parameter of the microscopic three - dimensional topography of a workpiece surface, referring to the spacing between tiny peaks and valleys on the machined part surface, which directly affects the performance and lifespan of the workpiece. Currently, roughness measurement methods are divided into two categories: contact methods and non - contact methods. The contact method uses a stylus to slide on the surface. Although it has a low cost, it may damage the surface microscopic topography. The non - contact method mainly uses optical methods to measure roughness through optical phenomena, avoiding surface damage. Among them, the machine vision method is widely used in roughness measurement due to its advantages of non - contact, high speed, and low cost.

[0003] Roughness measurement based on machine vision is divided into traditional artificial feature extraction methods and deep learning methods. The traditional method manually extracts speckle, grayscale, and texture features and combines them with random forest, support vector regression machine, or BP neural network for prediction. This method requires a small amount of data, has a lightweight model, and is suitable for online measurement, but is not suitable for high - precision offline measurement and is prone to introducing human errors. The deep learning method achieves higher precision and real - time performance through automatic feature extraction, but it relies on a large amount of data and a complex network structure. Although the precision is higher, these methods mainly establish a relationship between two - dimensional image features and roughness and do not fully utilize the microscopic three - dimensional topography information of the object.

[0004] In addition, the lighting conditions have an important impact on roughness measurement based on machine vision. Different light sources and incident angles have different reflections on the same surface, which can reveal different characteristic information of roughness. Therefore, reasonably using the image information under multiple light sources and multiple angles is crucial for improving the roughness prediction accuracy. However, the existing methods do not comprehensively analyze the relationship between the light source wavelength, incident angle, surface microscopic topography, and roughness.

[0005] In summary, the existing technologies mainly have the following deficiencies: The traditional method makes predictions based on designed features and has poor generalization. Although the deep learning method can perform automatic feature extraction, it is limited by the amount of data and model complexity. In addition, both of these methods do not comprehensively consider the relationship between the light source wavelength, incident angle, surface microscopic three - dimensional topography, and roughness.

[0006] After retrieval, the application publication number is CN115841468A, a method for detecting the surface roughness of strip steel based on multi-feature fusion, including: obtaining the laser speckle image of cold-rolled strip steel and constructing a sample data set; extracting the statistical features of the speckle image; constructing an image feature extraction model and training the constructed image feature extraction model, and using the trained image feature extraction model to extract the image features of the speckle image; based on the statistical features and the image features extracted by the image feature extraction model, obtaining the fusion features of the speckle image and constructing a multi-feature data set; constructing a strip steel surface roughness detection model and training the constructed strip steel surface roughness detection model with the multi-feature data set; based on the trained strip steel surface roughness detection model, detecting the surface roughness of strip steel. The present invention can realize the online detection of the surface roughness of strip steel and improve the detection accuracy of the surface roughness of cold-rolled strip steel.

[0007] This patent does not consider the different sensitivities of lasers with different wavelengths to surface roughness, nor does it introduce the influence of the change of the incident angle on the speckle characteristics. However, the present invention uses multi-wavelength laser speckle images to provide more comprehensive surface roughness information through multi-spectral feature fusion, and through the combination of light sources with different incident angles, surface information can be obtained from multiple angles, effectively enhancing the accuracy and stability of roughness detection. In addition, the feature fusion method is also different. This patent focuses on extracting multiple features from a single speckle image and weighting these features through an attention mechanism, while the present invention first performs multi-wavelength feature fusion to ensure the full integration of image features with different wavelengths, and then performs multi-angle feature fusion through a self-attention mechanism to ensure the best fusion of information at different angles, thereby significantly improving the feature expression ability and detection accuracy. Finally, it is worth noting that this patent uses a method combining a CNN model and a support vector machine, which is not an end-to-end method, while the present invention uses an end-to-end deep learning method to further improve the overall performance and accuracy of the model. Summary of the Invention

[0008] To solve the above problems, a new method combining multi-spectral laser speckle data and a photometric stereo vision model is proposed. Select N kinds of laser light sources with good monochromaticity, strong anti-interference ability, and sensitivity to microscopic details, and combine a telecentric lens and a high-resolution industrial camera to collect high-resolution speckle images generated by the change of the light source wavelength and the incident angle, and construct a multi-spectral laser speckle data set that comprehensively considers the light source wavelength, incident angle, surface three-dimensional morphology, and roughness. In the algorithm design, based on the principle of photometric stereo vision, analyze the relationship between the illumination angle and the surface microscopic morphology, and propose a convolutional neural network model for multi-angle feature aggregation. Through the joint input of multi-spectral and multi-angle information, learn complex high-dimensional feature relationships to achieve more accurate roughness measurement.

[0009] The technical solution of the present invention is as follows:

[0010] A roughness measurement method based on multi - spectral laser speckle images, which comprises the following steps:

[0011] Step 1, construct a multi - spectral laser speckle data set and collect laser speckle images of the surface of the object to be measured;

[0012] Step 2, perform channel stacking and feature fusion on the multi - spectral speckle images to generate multi - spectral fusion images;

[0013] Step 3, construct a single - angle feature extraction and fusion module to obtain surface feature information at a single angle;

[0014] Step 4, construct a multi - angle feature aggregation module, aggregate multi - angle surface feature information to obtain global illumination information, and finally calculate the roughness value of the object surface using the global illumination information.

[0015] Further, the construction of the multi - spectral speckle data set in Step 1 includes the following steps:

[0016] Rotate the workpiece object plane Π around the Z - axis in the space rectangular coordinate system obj to adjust the incident angle of the laser beam to θ; on the workpiece surface, move the workpiece along the Z - axis and Y - axis directions respectively with a fixed step size to adjust the laser irradiation position; switch laser light sources with different wavelengths, repeat the above steps, and respectively collect the speckle images formed after modulation by N wavelengths of laser on the workpiece surface;

[0017] Perform normalization on the collected image data to eliminate the image brightness difference caused by different laser light source intensities, and ensure that the features of multi - wavelength images are compared at the same scale; after normalization, divide them into different sub - folders according to different measured areas, group them according to different incident angles of the light source within the sub - folders, each angle contains N speckle images with different wavelengths, and the images of all angles form a multi - angle image sequence, and finally construct a multi - spectral laser speckle data set.

[0018] Further, the construction of the single - angle feature extraction and fusion module in Step 3 includes the following steps:

[0019] Multi - scale feature extraction: Use a CNN convolutional network fine - tuned with a hierarchical vision Transformer architecture based on a shifted window (SwinTransformer) as the backbone network to extract speckle features, texture, and color information from the multi - spectral fusion images; by inputting the features of different stages extracted by the backbone network into a pyramid network, process the images at multiple scales to achieve multi - scale feature extraction;

[0020] Feature fusion: The channel and spatial hybrid attention mechanisms are used to adjust the weights of low-level detailed information and high-level global information. After weighted calculation, the surface feature information under a single angle is generated.

[0021] Further, the multi-scale feature extraction includes the following steps:

[0022] First, a CNN convolutional network fine-tuned with a hierarchical vision Transformer architecture based on shifted windows (Swin Transformer) is used for multi-stage feature extraction. This network consists of 4 stages, each stage contains multiple feature extraction modules, and gradually extracts image features from low level to high level. Each feature extraction module uses depthwise separable convolution, combined with the GELU activation function and the LayerNorm layer normalization layer, to perform non-linear mapping and standardization processing on the features. As the stage progresses, the resolution of the feature map gradually decreases, while the number of channels gradually increases. The outputs of the 4 stages form a four-layer feature pyramid. Then, 1×1 convolution is used to adjust the number of channels horizontally to ensure that the number of channels of each layer of the feature map is the same, and the adjusted feature map is upsampled to make the size of each layer of the feature map unified and match the input size of the entire multi-scale feature network. Finally, the features of each layer are concatenated by channel to form a complete multi-scale feature representation.

[0023] Further, the feature fusion includes the following steps:

[0024] During the feature fusion process, the features are input into the spatial and channel hybrid attention mechanism for further optimization. First, the channel attention mechanism obtains the feature maps describing global information through global average pooling and global max pooling respectively, and adds these two feature maps element-wise. Then, the channel attention weights are generated through a shared fully connected layer to optimize the multi-spectral wavelength information, highlight the features corresponding to the key wavelengths, and suppress redundant information. Next, the spatial attention mechanism performs max pooling and average pooling on the feature map optimized by the channel attention to obtain two spatial information maps. After element-wise addition, the spatial attention weights are generated through a convolutional layer, which can adaptively adjust the importance of different spatial positions in the feature map. Finally, through the weighted fusion of the channel and spatial attention weights, each part of the feature map will be strengthened or suppressed. After double optimization, a more discriminative and multi-scale surface feature representation is obtained, effectively fusing multi-scale information and highlighting important speckle, texture, and spectral features.

[0025] Further, step 4 constructs a multi-angle feature aggregation module, including the following steps:

[0026] Through the multi-head self-attention mechanism, the multi-spectral fusion feature sequences {F 1 ,F2 ,…,F N} Calculate query Q, key K, and value V, where F N is the multi-spectral fusion feature at a certain angle output by the single-angle feature extraction and fusion module mentioned in step 3. Then, calculate the weighted attention score α ij between each feature image from angle j to angle i. Weight the value V j with these attention weights to obtain the fused feature

[0027] Transmit the global illumination information feature map after aggregating multi-angle features to a multi-layer perceptron MLP with L fully connected layers for non-linear mapping. Through layer-by-layer non-linear mapping, the MLP gradually abstracts and captures the relationship between the object surface from low-level texture, edge information to more complex shape, illumination changes, and these information and multi-angle features, so as to extract and integrate deeper features. Finally, output the roughness value r of the object surface, calculate the loss in combination with the true roughness value Ra, and use the smooth Huber loss function for supervised learning.

[0028] A roughness measurement device based on multi-spectral laser speckle images, comprising:

[0029] A displacement stage, a rotary stage, a lead screw stage, laser light sources of N wavelengths, a telecentric lens, a high-resolution industrial camera, and a control system; the displacement stage is installed in the center of the optical platform and can move along the Y-axis and Z-axis directions of the spatial rectangular coordinate system used by the device; the rotary stage is installed above the displacement stage and can rotate around the Z-axis of the device coordinate system; the lead screw stage is installed on the fixed bracket of the experimental table, and the moving direction of its slide is parallel to the Y-axis of the spatial coordinate system. The laser light sources of N wavelengths are fixed on the lead screw stage, and the slide control system can quickly switch between light sources of different wavelengths to ensure that the light source irradiates the center position of the workpiece surface; the telecentric lens and the high-resolution industrial camera are fixed above the laser light source, the optical axis of the lens is perpendicular to the surface of the workpiece to be measured, and the focal length is adjusted to ensure clear imaging; the control system is connected to the displacement stage, the rotary stage, the lead screw stage, and the industrial camera to coordinate the device actions and collect speckle images;

[0030] When the device is in use, the workpiece to be measured is fixed on the rotating table, so that the surface to be measured of the workpiece coincides with the rotating plane of the rotating table, and the central axis of the surface to be measured is aligned with the rotating axis of the rotating table; after the device is started, the control system first starts the displacement stage and the rotating table. The displacement stage moves the workpiece in the Y-axis and Z-axis directions to cover different regions, while the rotating table gradually changes the laser incident angle; the control system synchronously controls the lead screw stage to switch the light source, so that lasers of different wavelengths are respectively irradiated on the surface of the workpiece; the industrial camera works in cooperation with the telecentric lens to collect laser speckle images at different wavelengths and different incident angles in real time and transmit them to the computer for processing.

[0031] The advantages and beneficial effects of the present invention are as follows:

[0032] By combining multi-spectral laser speckle images and multi-angle light sources, the present invention innovatively constructs a multi-spectral laser speckle data set, which can comprehensively and accurately reflect the changes in surface micro-features. Compared with the existing methods that do not fully consider the influence of laser wavelength and incident angle on roughness measurement, the present invention comprehensively considers the complex relationship between light source wavelength, incident angle, surface three-dimensional morphology and roughness from both microscopic and macroscopic levels. Through the acquisition of multi-spectral laser speckle data, the present invention can provide richer multi-dimensional data support, effectively improve the accuracy of roughness measurement, and better adapt to the changes of complex surfaces. In terms of feature extraction, the present invention proposes a multi-angle feature aggregation neural network, which combines multi-spectral feature optimization and photometric stereo vision principle, and can improve the accuracy and stability of roughness measurement from both microscopic and macroscopic levels. At the microscopic level, through the multi-spectral composite image formed by channel stacking, combined with the channel attention mechanism to optimize the weight distribution of multi-spectral features, thus highlighting the sensitivity of different wavelengths to roughness features. At the macroscopic level, the surface micro-topography features are extracted by using the illumination changes at different angles, and the multi-angle information is globally optimized through the feature aggregation module, ensuring the stability and accuracy of roughness measurement. The present invention also adopts an end-to-end deep learning method, which can directly learn features from the original images, avoiding the cumbersome and limitations of the artificial design of feature extraction steps in traditional methods. Compared with traditional methods, the end-to-end deep learning method automatically optimizes the network parameters through training, enabling the model to more accurately capture roughness features and improving the accuracy and generalization ability of measurement results.

[0033] Through the above innovative steps, the present invention not only significantly improves the accuracy and stability in the field of roughness measurement, but also provides an efficient and intelligent technical solution that can adapt to the measurement requirements of different illuminations, angles and surface types. The application of this technology can provide more accurate surface quality assessment for fields such as precision manufacturing and quality inspection, and has broad practical application prospects. Description of the Drawings

[0034] Figure 1 This is the overall flowchart of a roughness measurement method based on multi - spectral laser speckle images provided by the preferred embodiment of the present invention.

[0035] Figure 2 This is a schematic diagram of the multi - spectral laser speckle image acquisition process of a roughness measurement method based on multi - spectral laser speckle images provided by an example of the method of the present invention;

[0036] Figure 3 This is a schematic diagram of a multi - angle feature aggregation neural network used in a roughness measurement method based on multi - spectral laser speckle images provided by the present invention.

[0037] Figure 4 This is a block diagram of the structural composition of a roughness measurement device based on multi - spectral laser speckle images provided by the present invention. Detailed implementation manners

[0038] Next, the technical solutions in the embodiments of the present invention will be clearly and detailedly described in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only a part of the embodiments of the present invention.

[0039] The technical solution of the present invention to solve the above - mentioned technical problems is:

[0040] The specific steps of the technical solution adopted in this scheme are as follows:

[0041] Step 1: Install and debug the acquisition device;

[0042] Step 2: Respectively use lasers with 1, 2, 3... N wavelengths as light sources, and use a telecentric lens and a high - resolution industrial camera to collect laser speckle images generated on the surface of the object to be measured due to the changes in the light source wavelength and the incident angle;

[0043] Step 3: Pre - process the collected data to construct a multi - spectral laser speckle data set;

[0044] Step 4: Stack the N multi - wavelength laser speckle images at a single angle according to channels to form a multi - channel composite image, and then generate a multi - spectral feature fusion image through convolution operations; construct a single - angle feature extraction and fusion module, and use a CNN convolutional network fine - tuned with a hierarchical vision Transformer architecture based on a shifted window (Swin Transformer) to perform multi - scale feature extraction on the multi - spectral fusion image at a certain angle generated in Step 2, then fuse the features at different scales, and finally obtain the surface feature information at a single angle; construct a multi - angle feature aggregation module, and integrate the surface feature information at different angles through the multi - angle feature aggregation module to generate a feature map containing global illumination information, and finally output the surface roughness value of the object through a multi - layer perceptron.

[0045] Preferably, as Figure 1 shown in the overall flowchart of a roughness measurement method based on multi-spectral laser speckle images provided by the method example of the present invention, specifically including the following steps:

[0046] Step 1.1: Initialize the acquisition device: Turn on the control system, start the control programs of the rotary table, displacement table and lead screw slide table, connect the industrial camera, and set the resolution and exposure parameters of the camera.

[0047] Step 1.2: Fix the light source and the camera: Select a laser light source of a certain wavelength and fix it to ensure stable emission of the light source. Start the camera and the laser light source, and test to acquire a sample image to verify the imaging quality.

[0048] Figure 2 shown is a schematic diagram of the multi-spectral laser speckle image acquisition process.

[0049] Step 2.1: Acquisition by angle: The rotary table starts from the starting angle and rotates step by step at a fixed stepping angle to the maximum angle. After stopping at each angle, trigger the industrial camera to acquire the speckle image under a laser of a certain wavelength. Step 2.2: Acquisition by position: After completing the data acquisition at each angle, control the displacement table to move the workpiece at a fixed step size to cover the entire surface range. For each position and angle, record the acquired image.

[0050] Step 2.3: Switch the light source: After completing the data acquisition of the light source of a certain wavelength, switch to a laser light source of other wavelengths through the lead screw slide table, and repeat the above acquisition process to ensure that the acquisition of N wavelengths is completed at all positions and angles.

[0051] Step 3: Perform normalization on the acquired image data to eliminate the image brightness differences caused by different incident angles and intensities of the laser light source, and ensure that the features of the multi-wavelength images are compared on the same scale. After completing the normalization, divide them into different sub-folders according to different measured areas, group them according to different image file names within the sub-folders to form a multi-spectral image sequence, and finally construct a multi-spectral laser speckle data set.

[0052] As Figure 3 shown is a schematic diagram of the multi-angle feature aggregation neural network used in a roughness measurement method based on multi-spectral laser speckle images provided by the present invention.

[0053] Step 4.1: Stack the N multi-wavelength laser speckle images at a single angle as input according to channels to form a multi-channel composite image, and then generate a multi-spectral feature fusion image through convolution operation;

[0054] Step 4.2: Use a CNN convolutional network fine-tuned with a hierarchical vision Transformer architecture (Swin Transformer) based on a shifted window for multi-stage feature extraction. As the stage progresses, the resolution of the feature map gradually decreases, while the number of channels gradually increases. The outputs of the 4 stages form a four-layer feature pyramid. Then, use 1×1 convolution to adjust the number of channels horizontally to ensure that the number of channels in each layer of the feature map is the same, and upsample the adjusted feature map to unify the size of each layer of the feature map to match the input size of the entire multi-scale feature network. Finally, concatenate the features of each layer by channel to form a complete multi-scale feature representation;

[0055] Step 4.3: Feature fusion, using a channel and spatial hybrid attention module to fuse the multi-scale features obtained in Step 4.2. First, calculate the channel attention weights through global average pooling (GAP) to optimize the multi-spectral wavelength information, highlight the features corresponding to the key wavelengths, and suppress redundant information. The formula is:

[0056]

[0057] where, is the input feature map, with dimensions (H, W, C), where: H is the height of the feature map, W is the width of the feature map, and C is the number of channels of the feature map. GAP(F (s) ) is the global average pooling operation on the input feature, W 1 is the weight matrix of the first fully connected layer, W 2 is the weight matrix of the second fully connected layer, δ is the ReLU activation function, σ is the Sigmoid activation function, mapping the weights to the interval [0, 1].

[0058] Then, calculate the spatial attention weights through the maximum pooling and average pooling of the feature map to adaptively adjust the importance of different spatial positions in the feature map, ensuring that the speckle, texture, and spectral features in the important regions are fully expressed. The formula is:

[0059] SA(F (s) ) = σ(Conv([MaxPool(F (s) ), AvgPool(F (s) )]))·F (s)

[0060] where, is the input feature map, with dimensions (H, W, C), where: H is the height of the feature map, W is the width of the feature map, and C is the number of channels of the feature map. MaxPool(F (s) ) is the maximum pooling operation on the input feature map, extracting the maximum activation value at each spatial position; AvgPool(F(s) ) performs an average pooling operation on the input feature map and calculates the average activation value of each spatial position; [MaxPool(F (s) ),AvgPool(F (s) )] is to concatenate the maximum pooling and average pooling results in the channel dimension to form a feature map with two channels. Conv is a two-dimensional convolution operation used to generate an attention weight map for each spatial position, and σ is a Sigmoid activation function that maps the weight to the [0,1] interval.

[0061] Finally, according to the calculated attention weights, the feature maps from different resolution levels of the feature pyramid are weighted and summed. While integrating the information at the micro and macro levels, the speckle, texture and spectral features are combined to generate a richer surface feature representation with multi-scale characteristics, and finally obtain the complete surface feature information at a single angle. Step 4.4: Multi-angle feature aggregation, the surface feature information at a single angle output by different branches is integrated through the multi-angle feature aggregation module to generate a feature map containing global illumination information, and finally output the surface roughness value of the object through a multi-layer perceptron, which specifically includes:

[0062] For each multispectral fusion feature {F 1 ,F 2 ,…,F N} Calculate the query Q, key K, value V, and then calculate the weighted attention score α between each feature image from angle j to angle i ij , through these attention weights value V j Perform weighted sum to obtain the fused features

[0063]

[0064] The global illumination information feature map after aggregating multi-angle features It is passed to a multi-layer perceptron (MLP) containing L fully connected layers for nonlinear mapping. Through layer-by-layer nonlinear mapping, the MLP gradually abstracts and captures the surface of the object from low-level texture and edge information to more complex shapes, illumination changes, and the relationship between this information and multi-angle features, thereby extracting and integrating deeper features. Finally, the roughness value r of the object surface is output, and the loss is calculated in combination with the roughness true value Ra, and the smooth Huber loss function is used for supervised learning:

[0065]

[0066] Among them, when the predicted value r and the true value r true The absolute error betweentrue When it is less than or equal to the threshold δ, the loss function adopts the form of quadratic error, expressed as At this time, more attention is paid to the precise optimization of small errors; when |r - r true | > δ, the loss function is converted into a linear form, expressed as to reduce the influence of outliers on the loss. The parameter δ is a hyperparameter, called the smoothing threshold, which is used to control the switching point from quadratic error to linear error.

[0067] As Figure 4 shown, a roughness measurement device based on multi-spectral laser speckle images provided by the present invention specifically includes the following components:

[0068] The present invention provides a roughness measurement device based on multi-spectral laser speckle, including a displacement stage, a rotary stage, a lead screw stage, laser light sources of N wavelengths, a telecentric lens, a high-resolution industrial camera, and a control system. The displacement stage is installed in the center of the optical platform and can move along the Y-axis and Z-axis directions of the experimental coordinate system. The rotary stage is installed above the displacement stage and can rotate around the Z-axis of the coordinate system to change the laser incident angle; the lead screw stage is fixed on the bracket of the experimental table, and the moving direction of the stage is parallel to the Y-axis of the coordinate system. The N laser light sources are fixed on it, and the stage control system can quickly switch the light sources to ensure that the light beam is always focused on the center of the surface to be measured of the workpiece; the telecentric lens and the high-resolution industrial camera are installed above the light source, and the optical axis of the lens is perpendicular to the workpiece surface. By adjusting the focal length, clear imaging is ensured. The control system is connected to the displacement stage, the rotary stage, the lead screw stage, and the industrial camera, and is responsible for coordinating the actions of the equipment and completing the acquisition of speckle images.

[0069] When the device is in use, first fix the workpiece to be measured on the rotary stage to ensure that the surface to be measured coincides with the rotation plane of the rotary stage, and the center of the surface is aligned with the rotation axis. After starting the device, the control system drives the displacement stage and the rotary stage in sequence. The displacement stage moves the workpiece position along the Y-axis and Z-axis directions to cover the entire surface to be measured, while the rotary stage gradually changes the laser incident angle on the workpiece surface. The control system synchronously drives the lead screw stage to switch the light sources, so that light sources of different wavelengths irradiate the workpiece surface in sequence. The industrial camera uses the telecentric lens to collect the speckle images at each light source and different angles in real time, and transmits the image data to the computer for storage and processing. After the collected multi-spectral and multi-angle speckle image data are classified and stored, they are used as inputs and passed to the roughness prediction model based on the multi-angle feature aggregation neural network, and finally the roughness prediction value of the workpiece surface is output.

[0070] The systems, devices, modules or units illustrated in the above embodiments may be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0071] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover a non-exclusive inclusion, such that a process, method, commodity or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, commodity or device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the presence of another identical element in the process, method, commodity or device comprising the element.

[0072] The above embodiments should be understood as being only for illustrative purposes of the present invention and not for limiting the protection scope of the present invention. After reading the content described in the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims

1. A roughness measurement method based on multispectral laser speckle image, characterized in that: The following steps are involved: Step 1: construct a multispectral laser speckle data set and collect laser speckle images on the surface of the object under test; Step 2, performing channel stacking and feature fusion on the multispectral speckle image to generate a multispectral fusion image; Step 3: construct a single-angle feature extraction and fusion module to obtain surface feature information at a single angle; Step 4: construct a multi-angle feature aggregation module to aggregate multi-angle surface feature information to obtain global illumination information, and finally use the global illumination information to calculate the roughness value of the object surface.

2. The roughness measurement method based on multispectral laser speckle image according to claim 1 is characterized in that: The step 1 constructs a multi-spectral laser speckle data set, including the following steps: In the spatial rectangular coordinate system, the object plane Π is rotated around the Z axis obj To adjust the incident angle of the laser beam to θ; on the workpiece surface, move the workpiece along the Z-axis and Y-axis directions with fixed steps to adjust the laser irradiation position; switch laser light sources with different wavelengths, repeat the above steps, and respectively collect speckle images formed by lasers of N wavelengths modulated on the workpiece surface; The collected image data are normalized to eliminate the image brightness differences caused by different laser light source intensities and ensure that the features of multi-wavelength images are compared at the same scale. After normalization, they are divided into different subfolders according to the measured areas. The subfolders are grouped according to the different light source incident angles. The same angle contains N speckle images of different wavelengths. Images at all angles form a multi-angle image sequence, and finally a multispectral laser speckle data set is constructed.

3. The roughness measurement method based on multispectral laser speckle image according to claim 1 is characterized in that: The step 3 constructs a single-angle feature extraction and fusion module, including the following steps: Multi-scale feature extraction: A CNN convolutional network fine-tuned with the shifted window-based hierarchical visual Transformer architecture, namely SwinTransformer, is used as the backbone network to extract speckle features, texture, and color information from multi-spectral fusion images. The features extracted at different stages by the backbone network are input into a pyramid network to process the image at multiple scales to achieve multi-scale feature extraction. Feature fusion: The channel and spatial mixed attention mechanism is used to adjust the weights of low-level detail information and high-level global information, and the surface feature information at a single angle is generated after weighted calculation.

4. The roughness measurement method based on multispectral laser speckle image according to claim 3 is characterized in that: The multi-scale feature extraction comprises the following steps: Firstly, a CNN convolutional network fine-tuned with a shifted window-based hierarchical visual Transformer architecture is used for multi-stage feature extraction. The network consists of four stages, each of which contains multiple feature extraction modules, which gradually extract image features from low-level to high-level. Each feature extraction module uses depthwise separable convolution, combined with GELU activation function and LayerNorm normalization layer, to perform nonlinear mapping and normalization on the features; as the stages progress, the resolution of the feature map gradually decreases, while the number of channels gradually increases, and the output of the four stages constitutes a four-layer feature pyramid; then, 1×1 convolution is used to adjust the number of channels horizontally to ensure that the number of channels of each layer of feature maps is consistent, and the adjusted feature maps are upsampled to make the size of each layer of feature maps uniform and match the input size of the entire multi-scale feature network; finally, the features of each layer are spliced ​​by channel to form a complete multi-scale feature representation.

5. The roughness measurement method based on multispectral laser speckle image according to claim 3 is characterized in that: The feature fusion includes the following steps: In the feature fusion process, the features are input into the spatial and channel hybrid attention mechanism for further optimization. First, the channel attention mechanism obtains feature maps describing global information through global average pooling and global maximum pooling, and adds the two feature maps element by element. Then, the channel attention weight is generated through a shared fully connected layer to optimize the multi-spectral wavelength information, highlight the features corresponding to the key wavelengths, and suppress redundant information. Next, the spatial attention mechanism performs maximum pooling and average pooling on the feature map after channel attention optimization to obtain two spatial information maps. After pixel-by-pixel addition, the spatial attention weights are generated through the convolutional layer, which can adaptively adjust the importance of different spatial positions in the feature map. Finally, through the weighted fusion of channel and spatial attention weights, each part of the feature map will be strengthened or suppressed. After double optimization, a more discernible and multi-scale surface feature representation is obtained, which effectively integrates multi-scale information and highlights important speckle, texture and spectral features.

6. The roughness measurement method based on multispectral laser speckle image according to claim 1, characterized in that: The step 4 constructs a multi-angle feature aggregation module, including the following steps: The multi-spectral fusion feature sequence {F1, F2, …, F N }Compute query Q, key K, value V, where F N It is the multi-spectral fusion feature of a certain angle output by the single-angle feature extraction and fusion module. Then, the weighted attention score α between each feature image from angle j to angle i is calculated. ij , through these attention weights value V j Perform weighted sum to obtain the fused features The global illumination information feature map after aggregating multi-angle features The image is passed to a multi-layer perceptron (MLP) containing L fully connected layers for nonlinear mapping. Through layer-by-layer nonlinear mapping, the MLP gradually abstracts and captures the surface information of the object from low-level texture and edge information to more complex shapes and illumination changes, as well as the relationship between these information and multi-angle features, thereby extracting and integrating deeper features. Finally, the roughness value r of the object surface is output, and the loss is calculated in combination with the roughness true value Ra, and the smooth Huber loss function is used for supervised learning.

7. A roughness measurement device based on multispectral laser speckle image, characterized in that: include: A translation stage, a rotating stage, a lead screw slide, a laser light source of N wavelengths, a telecentric lens, a high-resolution industrial camera and a control system; the translation stage is installed in the center of the optical platform and can move along the Y-axis and Z-axis directions of the spatial rectangular coordinate system used by the device; the rotating stage is installed above the translation stage and can rotate around the Z-axis of the device coordinate system; the lead screw slide is installed on a fixed bracket of the experimental table, and the moving direction of the slide is parallel to the Y-axis of the spatial coordinate system. The laser light source of N wavelengths is fixed on the lead screw slide, and the slide control system can quickly switch between light sources of different wavelengths to ensure that the light source irradiates the center position of the workpiece surface; the telecentric lens and the high-resolution industrial camera are fixed above the laser light source, the optical axis of the lens is perpendicular to the surface of the workpiece to be measured, and the focal length is adjusted to ensure clear imaging; the control system connects the translation stage, the rotating stage, the lead screw slide and the industrial camera to coordinate the movement of the equipment and collect speckle images; When the device is used, the workpiece to be measured is fixed on the rotating table so that the surface to be measured of the workpiece coincides with the rotating plane of the rotating table, and the central axis of the surface to be measured is aligned with the rotating axis of the rotating table; After starting the device, the control system first starts the translation stage and the rotation stage. The translation stage moves the position of the workpiece in the Y-axis and Z-axis directions to cover different areas, and the rotation stage gradually changes the laser incident angle; the control system synchronously controls the lead screw slide to switch the light source, so that lasers of different wavelengths irradiate the surface of the workpiece respectively; the industrial camera works with the telecentric lens to collect laser speckle images at different wavelengths and different incident angles in real time and transmit them to the computer for processing.

Images