A semantic segmentation method for disc brake vibration displacement measurement

Abstract

The present invention discloses a semantic segmentation method for disc brake vibration displacement measurement, belonging to the fields of visual vibration displacement measurement and computer vision. The present invention is used to obtain a corresponding segmentation mask for each frame of a disc brake vibration video after prediction by a semantic segmentation network model. The semantic segmentation network model includes an encoder and a decoder. The encoder uses Resnet101 as the network backbone, replaces the 7×7 convolution and maximum pooling with a downsampling module based on the Harr wavelet transform, and inserts a global-local feature extraction module into the original residual block of the Resnet101 network. The decoder uses a collaborative attention-based intra-scale feature interaction module and a Laplacian image-guided cross-scale feature fusion module. The present invention focuses on the balance between local and global information, effectively emphasizing edge information in the image, compensating for the lack of detail and local structural information in deep networks, improving the network model's positioning accuracy for the disc brake, and enhancing the semantic segmentation model's sensitivity and ability to capture subtle changes in the disc brake in dynamic environments.

Description

Technical Field

[0001] The invention relates to a semantic segmentation method for disc brake vibration displacement measurement, and belongs to the fields of visual vibration displacement measurement and computer vision. Background Art

[0002] Disc brakes are widely used in industry and transportation due to their efficient braking force. However, brake disc vibration caused by wear can cause a series of problems, including reduced braking performance, increased braking noise, and increased wear. These problems not only shorten component lifespan but can even lead to serious safety accidents. Therefore, accurately determining the vibration displacement curve of the disc brake's center point and identifying the fault type is crucial for measuring disc brake vibration displacement in complex scenarios.

[0003] There are two methods for measuring the vibration displacement of disc brakes: contact and non-contact. The contact method involves installing physical sensors on the brake surface, which is not only significantly affected by the environment but also alters the brake's dynamic characteristics. Non-contact measurement methods, such as ultrasonic, laser, and displacement sensors, are difficult to adapt to high-speed scenarios. Deep learning-based visual vibrometers can predict pixel points by learning from large amounts of data, thereby measuring the vibration displacement of disc brakes. Existing deep learning methods suffer from inaccurate boundary recognition, which can result in significant errors in the measurement of disc brake displacement. Summary of the Invention

[0004] The present invention provides a semantic segmentation method for disc brake vibration displacement measurement. The method aims to extract image features by using Resnet101 as the network backbone, adopting a downsampling strategy based on wavelet transform (HWTD) and a global-local feature extraction residual structure (GL-Bottleneck), and using a multi-scale feature fusion strategy guided by Laplacian image (LIFFN) to effectively integrate features from different coding layers to output the final segmentation result.

[0005] The technical solution of the present invention is:

[0006] According to a first aspect of the present invention, a semantic segmentation method for disc brake vibration displacement measurement is provided, comprising the following steps: obtaining disc brake vibration video data; each frame of the video is predicted by a semantic segmentation network model to obtain a corresponding segmentation mask; the semantic segmentation network model comprises an encoder and a decoder, wherein the encoder uses Resnet101 as the network backbone, and replaces the 7×7 convolution and maximum pooling with a downsampling module based on Harr wavelet transform, so that the output feature map has more detailed information of the disc brake in the image frame, and inserts a global-local feature extraction module into the original residual block of the Resnet101 network to improve the semantic segmentation network's ability to extract detailed information of the disc brake. The representation ability of node and local structure information is enhanced, and feature layers f2, f3, f4 and f5 are obtained through the encoder; in the decoder part, the intra-scale feature interaction module based on collaborative attention and the cross-scale feature fusion module guided by Laplacian image are adopted, and the feature layer f5 is input into the intra-scale feature interaction module based on collaborative attention. The spatial semantic information of the disc brake image is extracted from the feature layer through the intra-scale feature interaction module based on collaborative attention and the spatial semantic difference is reduced to obtain the feature layer P5. The original input image f1, feature layer f2, feature layer f3, feature layer f4, feature layer f5 and feature layer P5 are input into the cross-scale feature fusion module guided by Laplacian image to obtain the corresponding segmentation mask.

[0007] Furthermore, the encoder is specifically as follows: first, the input image f1 is processed twice by the Haar wavelet-based downsampling module, and after the first Haar wavelet-based downsampling module is processed, the feature layer f2 is obtained; the feature layer f2 is used as the input of the second Haar wavelet-based downsampling module, and the output of the second Haar wavelet-based downsampling module is sequentially passed through three residual blocks inserted into the global-local feature extraction module, and the output results of the three residual blocks inserted into the global-local feature extraction module are feature layers f3, f4 and f5 respectively.

[0008] Furthermore, the specific process of the Haar wavelet-based downsampling module is as follows: first, the resolution of the disc brake image is halved; then its signal is decomposed into two parts: a low-frequency L component and high-frequency HH, HV and HD components, and they are spliced ​​in the channel dimension; finally, a feature map is obtained through convolution, normalization and ReLU activation function processing.

[0009] Furthermore, the normalization is selected to use Group Normalization.

[0010] Furthermore, the residual block inserted into the global-local feature extraction module is specifically as follows: a global-local feature extraction module is added after the 1×1 and 3×3 convolutions of the original residual block part, and the global-local feature extraction module performs local adaptive average pooling on the input image to obtain local spatial information; it is then divided into two branches, the first branch performs global adaptive average pooling on the obtained local features, performs a convolution operation when the spatial dimension is reduced to 1, and then expands to the original input size of the module; the second branch adjusts the shape of the obtained local features, extracts deeper features through one-dimensional convolution, and then returns to the original input size of the module through shape adjustment; the features obtained by the two branches are weighted fused, and then restored to the same size as the input features through adaptive inverse average pooling to obtain an attention map; finally, the input features are multiplied by the generated attention map to obtain the output of the global-local feature extraction module.

[0011] Furthermore, the intra-scale feature interaction module based on collaborative attention is specifically as follows: the input feature layer f5 is average pooled, and the average values ​​are taken along the height and width dimensions respectively, and the feature maps of the two dimensions are divided into 4 groups of non-overlapping sub-features along the corresponding dimensions; then, a separable one-dimensional convolution with multi-receptive field depth sharing is applied to these sub-features, and they are spliced, normalized and activated respectively, and then adjusted back to the original dimensions to form a spatial attention map in two dimensions; and the obtained result is then multiplied by element by the initial input feature layer f5 of the intra-scale feature interaction module based on collaborative attention to obtain The weighted features that integrate global context dependencies and multi-semantic spatial information are average pooled and normalized on the features, and the dimensions are transformed through two-dimensional convolution to generate query q, key k and value v; then, the self-attention score between k and q is calculated along the channel dimension using the dot product method, and the channel attention relationship weight is obtained through softmax and dot product calculation is performed with v; finally, the output dimension of the self-attention is restored to the input shape, and after average pooling and activation function, the output feature is multiplied with the input "weighted features that integrate global context dependencies and multi-semantic spatial information" to obtain the feature layer P5.

[0012] Furthermore, the Laplacian image guided cross-scale feature fusion module is specifically: the feature layer f j+1 Perform 2 times upsampling and then combine with the feature layer f j Perform subtraction operations to obtain the Laplacian images L of each scale j; Among them, j = 1, 2, 3, 4; the feature layer P5 is fused with the feature layer f4 in the first fusion module to obtain the output feature layer x4; the feature layer P5 is subjected to the first convolution module five times to obtain the feature layer R5; the feature layer R5 is double-upsampled and stacked with x4 and L4 to obtain the feature layer P4; the feature layer P4 is subjected to the first convolution module four times and added with L4 to obtain R4; P4 is subjected to the first convolution module once, and then the first fusion module is used to obtain the output feature layer x3; the feature layer R4 is double-upsampled and stacked with x3 and L3 to obtain the feature layer P3; the feature layer P3 is subjected to the first convolution module three times and added with L3 to obtain R3; P3 is subjected to the first convolution module once, and then the first fusion module is used to obtain the output feature layer x3; f2 is fused to obtain the output feature layer x2; the feature layer R3 is upsampled twice and stacked with x2 and L2 to obtain the feature layer P2; the feature layer P2 passes through the first convolution module twice and is added with L2 to obtain R2; P2 passes through the first convolution module once and then passes through the first fusion module to obtain the feature layer x1 by fusing with the feature layer f1; the feature layer R2 is upsampled twice and stacked with x1 and L1 to obtain the feature layer P1; the feature layer P1 passes through three convolution modules and is added with L1 to obtain R1; R5 is upsampled twice and added with R4 to obtain D4; D4 is upsampled twice and added with R3 to obtain D3; D3 is upsampled twice and added with R2 to obtain D2; D2 is upsampled twice and added with R1 to obtain D1. D1 is the final output result of the semantic segmentation network.

[0013] According to a second aspect of the present invention, a processor is provided, wherein the processor is configured to execute operations including executing any one of the above-mentioned semantic segmentation methods for disc brake vibration displacement measurement.

[0014] The beneficial effects of the present invention are as follows: the present invention uses a high-speed industrial camera as an image acquisition medium and a disc brake in a high-speed video as an object of vibration displacement measurement, introduces a semantic segmentation method based on deep learning into the field of visual vibration measurement of disc brakes, and verifies the feasibility of deep learning methods in visual vibration measurement of disc brakes from multiple perspectives. Specifically: the present invention uses the classic semantic segmentation network Resnet101 as a basic framework, adopts a downsampling strategy based on wavelet transform (HWTD) and a global-local feature extraction residual structure (GL-Bottleneck) to extract image features, and uses a multi-scale feature fusion strategy (LIFFN) guided by Laplacian images to effectively integrate features from different coding layers to output the final segmentation result; according to the segmentation result, the coordinates of the center point of the image are extracted frame by frame, thereby more accurately measuring the vibration displacement offset. The new semantic segmentation method proposed in this invention focuses on the balance between local and global information, effectively emphasizes the edge information in the image, makes up for the lack of details and local structural information in the deep network, improves the network model's positioning accuracy for the disc brake, and improves the semantic segmentation network model's sensitivity and ability to capture subtle changes in the disc brake in a dynamic environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a workflow diagram;

[0016] Figure 2 This is the encoder structure diagram for semantic segmentation;

[0017] Figure 3 It is a downsampling module based on Haar wavelet transform;

[0018] Figure 4 Schematic diagram of intra-scale feature interaction modules based on collaborative attention;

[0019] Figure 5 Generate a diagram of the process for the Laplacian image;

[0020] Figure 6 This is a diagram of the semantic segmentation network model that introduces the Laplacian image-guided multi-scale feature fusion module;

[0021] Figure 7 Comparison chart of vibration displacement curves of different algorithms;

[0022] Figure 8 This is a comparison chart of axis trajectories of different algorithms;

[0023] Figure 9 The following is a comparison chart of amplitude-frequency curves of different algorithms. DETAILED DESCRIPTION

[0024] To make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. It should be noted that, in the absence of conflict, the embodiments in this application and the features in the embodiments can be combined with each other in any way.

[0025] Example 1: Figure 1-9 As shown, according to a first aspect of an embodiment of the present invention, a semantic segmentation method for disc brake vibration displacement measurement is provided, comprising: obtaining disc brake vibration video data; obtaining a corresponding segmentation mask for each frame of the video after prediction by a semantic segmentation network model; the semantic segmentation network model comprises an encoder and a decoder, wherein Resnet101 is used as the network backbone in the encoder, and the 7×7 convolution and maximum pooling therein are replaced with a downsampling module (HWTD) based on Harr wavelet transform, so that the output feature map has more detailed information of the disc brake in the image frame, and a global-local feature extraction module (GL-Bottleneck) is inserted into the original residual block of the Resnet101 network to improve the semantic segmentation network's ability to represent disc brake details and local structural information, and feature layers f2, f3, f4 and f5 are obtained through the encoder; and a Laplace Image-guided Feature Fusion network (Laplace Image-guided Feature Fusion) is used in the decoder part. Abstract: A novel LIFFN (Life-Long Response Network) is proposed, which includes a collaborative attention-based intra-scale feature interaction module (SAIFI) and a Laplacian image-guided cross-scale feature fusion module (LICFF). The feature layer f5 is input into the collaborative attention-based intra-scale feature interaction module. The spatial semantic information of the disc brake image is extracted from the feature layer by the collaborative attention-based intra-scale feature interaction module and the spatial semantic difference is reduced to obtain the feature layer P5. The original input image f1, feature layer f2, feature layer f3, feature layer f4, feature layer f5 and feature layer P5 are input into the Laplacian image-guided cross-scale feature fusion module to obtain the corresponding segmentation mask.

[0026] Furthermore, if Figure 2As shown in the figure, the Resnet101 backbone feature extraction network is constructed, and the 7×7 convolution and maximum pooling are replaced with a downsampling module based on Haar wavelet transform (HWTD). A global-local feature extraction module (GL-Bottleneck) is inserted into the original residual block of the Resnet101 network to reduce the spatial resolution of the image while retaining more effective semantic information. The encoder specifically: first, the input image f1 is processed twice by the Haar wavelet-based downsampling module. After the first Haar wavelet-based downsampling module is processed, the feature layer f2 is obtained; the feature layer f2 is used as the input of the second Haar wavelet-based downsampling module, and the output of the second Haar wavelet-based downsampling module is sequentially passed through three residual blocks inserted into the global-local feature extraction module. The output results of the three residual blocks inserted into the global-local feature extraction module are feature layers f3, f4 and f5 respectively.

[0027] Furthermore, Resnet101 is used as the backbone network to perform preliminary feature extraction on the input disc brake image. First, the input image f1 is downsampled twice based on Haar wavelet. After the first downsampling based on Haar wavelet, the feature layer f2 is obtained. Figure 3 As shown in the figure, the specific process of the Haar wavelet-based downsampling strategy is as follows: first, the resolution of the disc brake image is halved; then, its signal is decomposed into two parts: low-frequency (L) component and high-frequency components (HH, HV, and HD), and these are spliced ​​in the channel dimension to expand the channel. Finally, the feature map is obtained through convolution, normalization, and ReLU activation function processing, where Group Normalization is selected for normalization. Figure 2As shown, the residual block adds a global-local feature extraction module after the 1×1 and 3×3 convolutions. This module performs local adaptive average pooling on the input image to obtain local spatial information. It then splits into two branches. The first branch performs global adaptive average pooling on the obtained local features, performs a convolution operation when the spatial dimension is reduced to 1, and then expands to the original input size of the module. The second branch reshapes the obtained local features, extracts deeper features through one-dimensional convolution, and then reshapes them back to the original input size of the module. The features obtained from the two branches are weighted fused and restored to the same size as the input features through adaptive inverse average pooling to form an attention map. Finally, the input features are multiplied with the generated attention map to obtain the output of the global-local feature extraction module. The residual block first performs 1×1 dimensionality reduction on the image in terms of the number of channels, and then extracts features from the image using 3×3 convolution. The extracted features are multiplied by the generated attention map through the global-local feature extraction module, and the dimensionality is restored using 1×1 convolution. Finally, the extracted features are added to the input image. The output results of the three residual blocks are feature layers f3, f4 and f5 respectively.

[0028] In the above, Group Normalization is chosen for normalization. The purpose of using it is: considering that when the batch size is small, the calculated mean and variance of BatchNormalization are not accurate enough, resulting in poor normalization effect. Although LayerNormalization does not depend on the batch size, it ignores the information between channels and only normalizes between neurons in the same layer. Therefore, the present invention chooses to use Group Normalization, which not only preserves the dependency between channels, but also can maintain good normalization effect under small batches.

[0029] Furthermore, the decoder part adopts a Laplacian image guided multi-scale feature fusion strategy (LIFFN), which includes a collaborative attention-based intra-scale feature interaction module (SAIFI) and a Laplacian image guided cross-scale feature fusion module (LICFF) to enhance the feature extraction of the image.

[0030] Furthermore, if Figure 4As shown in the figure, the extracted effective feature layer f5 is input into the scale feature interaction module based on collaborative attention, which is designed to guide channel attention learning: first, the input feature f5 is average pooled, and the average values ​​are taken along the height and width dimensions respectively. The feature maps of the two dimensions are divided into 4 groups of non-overlapping sub-features along the corresponding dimensions; then, the separable one-dimensional convolution DWConv1d with multi-receptive field depth sharing is applied to these sub-features, and they are spliced, normalized and activated respectively, and then adjusted back to the original dimension to form a spatial attention map in two dimensions; then the obtained result is compared with the initial input feature of the scale feature interaction module based on collaborative attention. f5 obtains the weighted features that integrate global context dependencies and multi-semantic spatial information through element-wise multiplication, performs average pooling and normalization on the features, and performs dimensionality transformation through two-dimensional convolution DWConv to generate query (q), key (k) and value (v); then, the self-attention score between k and q is calculated along the channel dimension using the dot product method, and the channel attention relationship weight is obtained through softmax and the dot product is calculated with v; finally, the output dimension of the self-attention is restored to the input shape, and after average pooling and activation function, the output feature is multiplied with the input "weighted features that integrate global context dependencies and multi-semantic spatial information" to obtain the feature layer P5.

[0031] Furthermore, the results of the feature layers f1, f2, f3, f4 and f5 and the collaborative attention-based intra-scale feature interaction module are input into the Laplacian image-guided cross-scale feature fusion module.

[0032] First, if Figure 5 As shown, the method for calculating the Laplace residuals of each level of the input disc brake RGB image is as follows: j traverses 1 to 4 and takes the feature layer f j+1 Perform 2 times upsampling and then combine with the feature layer f j Perform subtraction operations to obtain the Laplacian images L of each scale j .

[0033] L j =f j -Up(f j+1 ),j=1,2,3,4

[0034] From the feature extraction part, we can know that Up is a 2-fold upsampling function, and the feature layer f j+1 The size of f j 1 / 2 times.

[0035] Then, if Figure 6As shown in the figure, the feature layer P5 is normalized, activated, upsampled, and convolved in the first fusion module, then fused with the feature layer f4, and then outputs the feature layer x4 through one convolution. The feature layer P5 passes through the first convolution module five times, that is, after five times of normalization, activation, weight normalization, and convolution, the feature layer R5 is obtained.

[0036] The feature layer R5 is upsampled twice and then stacked with x4 and L4 to obtain the feature layer P4; the feature layer P4 undergoes four first convolution modules, namely normalization, activation function, weight normalization and convolution, and then is added with L4 to obtain R4; P4 undergoes one first convolution module, and then passes through the first fusion module, where it undergoes normalization, activation function, upsampling and convolution, and is fused with the feature layer f3, and then undergoes one more convolution to output the feature layer x3;

[0037] The feature layer R4 is upsampled twice and then stacked with x3 and L3 to obtain the feature layer P3; the feature layer P3 passes through the first convolution module three times and is added with L3 to obtain R3; P3 passes through the first convolution module once, and then passes through the first fusion module once, where it undergoes normalization, activation function, upsampling and convolution processing, and is fused with the feature layer f2, and then passes through one more convolution to output the feature layer x2;

[0038] The feature layer R3 is upsampled twice and then stacked with x2 and L2 to obtain the feature layer P2; the feature layer P2 passes through the first convolution module twice and is added with L2 to obtain R2; P2 passes through the first convolution module once, and then passes through the first fusion module once, where it undergoes normalization, activation function, upsampling and convolution processing, and is fused with the feature layer f1, and then passes through another convolution to output the feature layer x1;

[0039] The feature layer R2 is upsampled twice and then stacked with x1 and L1 to obtain the feature layer P1; the feature layer P1 passes through three convolution modules and is added with L1 to obtain R1.

[0040] In the above, about R j The expression of R is as follows: j =B j ([x j ,L j ,Up(R j+1 )])+L j ,j=1,2,3,4; where Bj means stacking and Up means double upsampling.

[0041] Finally, R5 is upsampled twice and added to R4 to obtain D4; D4 is upsampled twice and added to R3 to obtain D3; D3 is upsampled twice and added to R2 to obtain D2; D2 is upsampled twice and added to R1 to obtain D1. D1 is the final output result of the semantic segmentation network.

[0042] In the above, about D j The expression is as follows:

[0043] This process proceeds layer by layer, mapping the features back to the size of the original disc brake image from coarse to fine, successfully recovering the local details and global layout of the disc brake.

[0044] In specific applications, the model proposed in the present invention can be trained first, and the semantic segmentation model of the disc brake is added with the Focal-Loss loss function during training.

[0045] The deep learning algorithm used in this paper was run on a desktop computer (equipped with an Intel Core i7-13700K processor, NVIDIA GeForce RTX 4080 graphics card, and 32G video memory) equipped with a unified operating environment (Windows 10, Cuda 11.8, Pytorch 2.0.1, and torch-vision 0.15.2).

[0046] This disc brake vibration displacement dataset was generated using disc brake vibration data captured with a high-speed camera. The constructed data acquisition system uses a high-speed industrial camera as the image data sensor, and an eddy current sensor as the actual vibration displacement sensor, with both synchronously acquiring signals. The data acquisition system primarily consists of three components: a vibration test and control bench, which includes a rotor test bench, disc brake, and power supply equipment; a vibration measurement system, and an image acquisition system. The vibration test and control bench includes two eddy current sensors with a two-millimeter range, two signal acquisition cards, a DH5923 dynamic signal acquisition and analysis instrument, and a desktop computer equipped with specialized DHDAS software. The image acquisition system includes a Thousand Eye Wolf M220M high-speed camera, a corresponding light source, and a laptop computer equipped with high-speed acquisition software. Before data acquisition begins, the signal acquisition card must be connected to the DH5923 dynamic signal acquisition and analysis instrument and the eddy current sensor connected to the desktop computer. The eddy current sensor probes, used to measure radial and tangential displacement, must be aligned with the horizontal and vertical centerlines of the disc brake. The eddy-current sensor's signal sampling rate was set to 2000 fps. A high-speed camera was placed 1500 mm in front of the disc brake, with a resolution of 960×960 and a frame rate of 2000 fps. The light source was placed diagonally in front of the disc brake, with the brightness set to 75%. The power supply's voltage and current were adjusted to ensure smooth rotor operation. A synchronized sampling program enabled the high-speed industrial camera and eddy-current sensor to simultaneously capture the disc brake's image sequence and voltage displacement signals.

[0047] Furthermore, 500 frames were randomly selected from the captured video and annotated using Labelme, the annotation software. The coordinates of the disc brake rotor bounding box were annotated. The training and validation datasets comprised 90% and 10% of the total disc brake vibration displacement dataset, respectively. A semantic segmentation network model was trained using the training dataset and then evaluated using the test dataset. The optimal weight file was obtained and loaded into the semantic segmentation network model.

[0048] Furthermore, the collected data is input into the trained semantic segmentation model for prediction, such as Figure 6 As presented, a segmentation mask belonging to the disc brake class is extracted and the coordinates of each pixel are obtained. These coordinates are then averaged to calculate the coordinates of the center point of the segmentation mask for each frame. The average value of the disc brake center point coordinates for each frame is used as a reference value to ensure that all displacement offsets in the vibration signal conform to the zero-mean distribution requirement. By calculating the displacement offset between each frame and this reference value, the vibration displacement curve of the disc brake during rotation can be obtained.

[0049] Furthermore, the present invention uses three evaluation indicators to evaluate the performance of the semantic segmentation network model: recall rate (Recall), category mean pixel accuracy (mean PixelAccuracy, mPA) and mean intersection over union (mIoU). The calculation formula is as follows:

[0050]

[0051] Among them, TP represents the correctly predicted positive sample, FP represents the incorrectly predicted positive sample, TN represents the correctly predicted negative sample, and FN represents the incorrectly predicted negative sample. Disc Intersection over Union (IoU) of the target representing the disc brake background The intersection-over-combination ratio (IoU) of the background representing the disc brake; PA Disc Represents the pixel accuracy of the disc brake target, PA background Pixel accuracy of the background representing the disc brake.

[0052] Furthermore, frequency domain indicators and time domain indicators are two commonly used indicators in vibration detection and evaluation. Therefore, the present invention integrates time domain and frequency domain indicators to construct an evaluation system for visual vibration measurement tasks, thereby more accurately and comprehensively reflecting the changes in the motion state of the disc brake.

[0053] Time-domain metrics evaluate the vibration signal over time, reflecting its changes along the time axis. This paper selects waveform factor error (WFE), clearance factor error (CFE), and normalized displacement root mean square error (d-NRMSE) as time-domain evaluation metrics.

[0054] The calculation process of WFE, CFE, and d-NRMSE is as follows:

[0055]

[0056] WFE=|WF1-WF2|

[0057] Among them, n is the number of sampling points, x i is the displacement data of the i-th sampling point in the vibration signal, WF1 and WF2 are the waveform factors of the vibration displacement signal measured by the semantic segmentation method of the present invention and the eddy current sensor, respectively.

[0058]

[0059] CFE=|CF1-CF2|

[0060] Among them, CF1 and CF2 are the margin factors of the semantic segmentation method of the present invention and the vibration displacement signal measured by the eddy current sensor, respectively.

[0061]

[0062] in, and x i are the displacement data of the i-th sampling point in the vibration displacement signal measured by the semantic segmentation method of the present invention and the eddy current sensor, respectively, max and x min The d-NRMSE values ​​are the maximum and minimum values ​​of the true displacement measured by the eddy current sensor, respectively. The smaller the d-NRMSE value, the better the fit between the vibration displacement curve predicted by the network model and the true displacement curve, and the better it meets the actual needs of this experimental scenario.

[0063] Frequency domain indicators are a method for evaluating the frequency components of vibration signals. The present invention selects center of gravity frequency error (CGFE), frequency-root mean square error (f-RMSE), and fundamental frequency amplitude error (FFAE) as frequency domain evaluation indicators.

[0064] The calculation process of CGFE, f-RMSE, and FFAE is as follows:

[0065]

[0066] CGFE=|CGF1-CGF2|

[0067] Where m is the number of main frequency components, f k is the frequency value of the kth frequency component, and s(k) is the corresponding amplitude.

[0068]

[0069] f-RMSE=|f-RMS1-f-RMS2|

[0070] Among them, f-RMS1 and f-RMS2 are the frequency root mean square of the vibration displacement signal measured by the semantic segmentation method and the eddy current sensor, respectively.

[0071] FFAE=|FFA1-FFA2|

[0072] Among them, FFA1 and FFA2 are the fundamental frequency amplitudes of the vibration displacement signals measured by the semantic segmentation method and the eddy current sensor, respectively.

[0073] Furthermore, to evaluate and understand the effectiveness and contribution of this method to the overall performance of the semantic segmentation network model and disc brake vibration displacement measurement, five sets of ablation experiments were conducted on a self-made dataset, using the UNet network as the baseline network model (Baseline). Each set was based on the previous set. In Table 1, "+Layeradjust" represents adding "Layer adjust" to the "Baseline," replacing the ResNet50 in the UNet encoder with a ResNet101; "+LIFFN" represents adding "LIFFN" to the "Baseline+Layer adjust," adding the LIFFN module to the encoder; "+HWTD" represents adding HWTD to the "Baseline+Layer adjust+LIFFN," using the HWTD module in the encoder; and "+GL-Bottleneck" represents adding GL-Bottleneck to the "Baseline+Layer adjust+LIFFN+HWTD," using the GL-Bottleneck module in the decoder. In the ablation experiment, the present invention adjusted the number of encoder layers and used a deeper network structure to adapt to the disc brake micro-vibration measurement task that requires high precision and complex feature extraction. The detailed results of the ablation experiment are shown in Table 1.

[0074] Table 1 Statistics of ablation experiment results

[0075]

[0076] From the ablation experiment results in Table 1, the model proposed in the present invention can achieve the network performance of the existing benchmark model. Furthermore, in vibration analysis, time domain indicators are used to evaluate the time domain characteristics in the vibration signal, such as the degree of impact and periodic vibration; while frequency domain indicators are used to evaluate the frequency components of the vibration signal, such as resonance, natural frequency and specific fault frequency. Combining these two indicators can more accurately diagnose the health status of the disc brake, detect problems early and take maintenance measures. Among the evaluation indicators of the visual vibration measurement task of the present invention (NRMSE, WFE, CGFE and f-RMSE), only the WFE in the y direction is slightly lower than the baseline model, and the remaining time domain and frequency domain indicators have achieved significant improvements. This shows that the adjustment of the number of network layers significantly improves the accuracy of vibration displacement measurement.

[0077] Specifically, as shown in Table 1, after introducing LIFFN into the semantic segmentation model, the d-NRMSE indicator decreased by 1.5% in the x-direction and 0.09% in the y-direction; the CGFE indicator decreased by 32.0370347 in the x-direction and 4.63383015 in the y-direction; and the f-RMSE indicator decreased by 30.6498079 in the x-direction and 2.73538304 in the y-direction. This further enhances the overall performance of the network and the accuracy of vibration displacement measurement.

[0078] After introducing HWTD into the semantic segmentation model, the Recall indicator increased to 99.67%; the mPA indicator increased to 99.83%; the mIoU indicator increased to 99.65%; and the NRMSE indicator decreased to 0.049 and 0.0259 in the x and y directions respectively.

[0079] Incorporating GL-Bottleneck into the semantic segmentation model demonstrated excellent performance across all vibration displacement measurement evaluation metrics. Compared to the baseline model, the final algorithm achieved reductions of 5.7% in the NRMSE, 69.51427% in the CGFE, and 60.5425635% in the x-direction; and 3.23% in the NRMSE, 0.014596783% in the WFE, 30.41015573% in the CGFE, and 31.15941723% in the y-direction. GL-Bottleneck improves the model's focus on detailed structural information in disc brake images by balancing global and local information.

[0080] The improvement of the indicators shows that the innovative part of the semantic segmentation algorithm proposed in the present invention is feasible and effective in improving the vibration displacement measurement accuracy of the disc brake.

[0081] Furthermore, to verify the performance of our method in semantic segmentation and vibration displacement measurement accuracy, we conducted a comprehensive performance comparison between our proposed semantic segmentation algorithm and five currently representative semantic segmentation algorithms (FCN, DeepLabV3+, PSPNet, HRNet, and SegFormer) using the same evaluation system. To ensure the fairness of the comparative test, all algorithms were trained for 300 rounds on a custom dataset, with a batch size of 4 and all other parameters maintained at their default settings.

[0082] This method uses a custom dataset as input to the various semantic segmentation algorithms mentioned above for training and prediction. It then performs time-domain and frequency-domain analysis on the disc brake vibration displacement data, regressing the x- and y-axis displacement curves, axis trajectory plots, and frequency spectrograms. The predictions from the various semantic segmentation algorithms are then compared with standard vibration displacement signals collected by eddy current sensors for qualitative comparison.

[0083] like Figure 7 As shown ( Figure 7 The first to sixth rows are FCN, DeepLabV3+, PSPNet, HRNet, SegFormer, and the present invention, respectively, with the left side of each row being the x-direction and the right side being the y-direction. The vibration displacement curve in the y-direction shows a better fitting effect than that in the x-direction as a whole. This is because the rotor is affected by its own gravity when rotating, which makes the displacement change in the y-direction greater than that in the x-direction. Therefore, the semantic segmentation algorithm has relatively little difficulty in capturing the position change in the y-direction. The overall performance of FCN is the worst. Although it can roughly fit the vibration period, there is a lot of noise in its vibration displacement signal, which causes spikes and burrs to appear in the curve, and the peak and trough are significantly offset, which ultimately makes the overall vibration displacement amplitude lower than the standard signal. Although the SegFormer, PSPNet, and DeepLabV3+ networks are better than FCN in fitting the vibration displacement curve, they are still interfered by a small amount of noise information, and the measurement accuracy at the displacement peak is low. Furthermore, the displacement curves derived from regression show that to the right of SegFormer's y-direction displacement curve, the x-direction displacements of PSPNet below the mean, and the x-direction displacements above and below the mean of DeepLabV3+, are significantly lower than the actual displacement values, failing to effectively fit the standard signal. Compared to previous algorithms, the HRNet algorithm performs well in fitting the vibration period to the true displacement curve, with a smoother overall curve. However, performance at peaks and troughs still needs improvement, failing to effectively approximate the true amplitude.

[0084] By comparing the vibration displacement curves, it can be found that the vibration displacement signal fitted by the method proposed in the present invention has less interference information and the curve is smoothest, successfully overcoming the shortcomings of peaks and troughs that are common in other comparison algorithms, thereby being as close to the actual displacement data as possible.

[0085] Furthermore, the vibration displacement data in the x and y directions are synthesized to obtain the axis trajectory diagram to represent the actual dynamic path of the disc brake, such as Figure 8 As shown ( Figure 8The first row from left to right is FCN, DeepLabV3+, PSPNet, and the second row from right is HRNet, SegFormer, and the present invention). The waveform trajectory regressed from the displacement measured by the eddy current sensor is elliptical. The main reason is that the support stiffness in the x and y directions is asymmetric or the force is unbalanced, which is consistent with the experimental setting. It can be seen that the trajectory fitted by the FCN algorithm is no longer an elliptical shape, but an irregular and chaotic closed curve. Although the trajectories of other algorithms are approximately elliptical, there is a lot of noise, and it is almost wrapped in the actual axis trajectory, which is quite different from the standard trajectory. The algorithm proposed in the present invention has the highest degree of fit with the real axis trajectory and the least noise information.

[0086] Furthermore, the fast Fourier transform is used to perform spectrum analysis on the vibration signal of the disc brake, and the frequency components such as the fundamental frequency and each harmonic component contained in the signal can be clearly observed. At around 10Hz, the spectrum graph shows a peak, and the peak has the highest intensity in the signal, representing the basic vibration frequency of the rotor, that is, the fundamental frequency, which is directly related to the rotation speed of the rotor. The amplitude of the harmonics is significantly smaller than the fundamental frequency, which indicates that the vibration of the disc brake is mainly concentrated on the rotation frequency and is not subject to serious vibration interference. However, the disc brake has a slight uneven mass distribution. This unbalanced state will cause additional vibration frequencies, and these harmonic frequencies are usually integer multiples of the fundamental frequency. Based on Figure 9 The comparison results of the spectrum graph show that (( Figure 9 The first column, from top to bottom, is FCN-x, FCN-y, HRNet-x, and HRNet-y; the second column, from top to bottom, is DeepLabV3+-x, DeepLabV3+-y, SegFormer-x, and SegFormer-y; the third column, from top to bottom, is PSPNet-x, PSPNet-y, the present invention-x, and the present invention-y). The fundamental frequencies of each algorithm are generally consistent with the standard signal, but there are significant differences in the degree of fit of the amplitude. The amplitude of the algorithm proposed by the present invention is closer to the standard eddy current signal and can more accurately reflect the energy or power of the frequency component in the vibration signal. In addition, it can be clearly seen from the spectrum diagram that a large number of irrelevant frequency components appear in the spectrum diagram (especially in the x direction) of each comparison algorithm, while the method proposed by the present invention performs best at each harmonic, with almost no such situation occurring. These erroneous harmonic components will largely cause the frequency center of gravity to shift to the right, and the root mean square frequency will be higher than the actual value, thereby seriously misleading the observer's accurate judgment of the health status of the disc brake.

[0087] Furthermore, based on the ablation experiment evaluation indicators, this paper adds CFE and FFAE to more comprehensively analyze the time and frequency domain performance of the proposed method, thereby performing a quantitative analysis of different algorithms. The comparison results are shown in Table 2.

[0088] Table 2: Comparison results of different algorithms

[0089]

[0090] In Table 2, from the perspective of semantic segmentation network evaluation indicators, the mPA and mIOU indicators of the proposed method perform best, among which the Recall indicator is as high as 99.65%; from the vibration displacement measurement results, the vibration displacement measurement performance of the algorithm of the present invention in the x and y directions is almost overwhelmingly superior to other semantic segmentation algorithms in all indicators in the time domain and frequency domain.

[0091] Specifically:

[0092] FCN performed the worst in vibration displacement measurement, with errors relative to the standard signal significantly higher than those of other algorithms across all evaluation metrics. This is due to the simple deconvolution upsampling used in the decoder, which results in blurry and smoothed segmentation results and an inability to effectively identify relationships between pixels. Furthermore, FCN's inability to integrate long-range dependencies and contextual information also resulted in an inability to effectively segment disc brakes.

[0093] SegFormer generally performs poorly in the x and y directions, particularly in the d-NRMSE (7.14% and 6.78%), CGFE (212.0786695 and 131.6053557%), and f-RMSE (250.4417194 and 149.1209699%). SegFormer uses the Transformer architecture to capture long-range dependencies, but its ability to capture local features is often inferior to CNNs for very subtle changes.

[0094] PSPNet also differs significantly from the true displacement signal in the time and frequency domains, with a d-NRMSE in the x-direction reaching 0.0857. PSPNet uses a pyramid pooling module to capture global contextual information at different scales, which is helpful for understanding large regions. However, the pooling operation itself loses some spatial information and often overlooks key local features for subtle changes.

[0095] DeepLabV3+ achieves a high FFAE of 0.17592 in the x-direction. This is because, while DeepLabV3 can extract multi-scale information through dilated convolution, the decoder does not fully integrate the multi-scale feature maps, resulting in a low level of segmentation precision in the final semantic segmentation map. Since disc brake vibration displacement is typically small, this network design can easily lead to blurred edges, affecting the accuracy of displacement measurement.

[0096] The method of the present invention effectively overcomes the shortcomings of the aforementioned comparative algorithms and achieves the most outstanding results in vibration displacement measurement accuracy. This is due to a downsampling method that effectively preserves high-resolution features, excellent global-local feature information coordination, and a multi-scale fusion strategy guided by Laplace residuals that is rich in edge feature information. This method effectively overcomes the shortcomings of the aforementioned comparative algorithms and achieves the most significant results in vibration displacement measurement accuracy.

[0097] In summary, the proposed method not only has excellent network performance but also performs well in visual vibrometer tasks. Compared with the most representative semantic segmentation algorithms, the proposed method demonstrates significant advantages across multiple key evaluation metrics, demonstrating its ability to handle subtle changes under dynamic conditions and providing a solid data foundation for subsequent fault diagnosis and maintenance decision-making.

[0098] According to a second aspect of an embodiment of the present invention, a processor is provided, wherein the processor is configured to execute an operation, wherein the operation includes executing any one of the above-mentioned semantic segmentation methods for disc brake vibration displacement measurement.

[0099] An embodiment of the present invention provides a device comprising a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, the following steps are implemented: obtaining disc brake vibration video data; and obtaining a corresponding segmentation mask for each frame of the video after prediction by a semantic segmentation network model.

[0100] The specific embodiments of the present invention are described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in this field without departing from the scope of the present invention.

Claims

1. A semantic segmentation method for disc brake vibration displacement measurement, characterized in that: The following steps are involved: Acquire disc brake vibration video data; Each frame of the video is predicted by the semantic segmentation network model to obtain the corresponding segmentation mask; the semantic segmentation network model includes an encoder and a decoder. The encoder uses Resnet101 as the network backbone, and replaces the 7×7 convolution and maximum pooling with a downsampling module based on Harr wavelet transform, so that the output feature map has more detailed information of the disc brake in the image frame, and inserts a global-local feature extraction module into the original residual block of the Resnet101 network to improve the semantic segmentation network's ability to represent disc brake details and local structural information. The feature layers f2 and f3 are obtained through the encoder. , f4 and f5; in the decoder part, a scale feature interaction module based on collaborative attention and a cross-scale feature fusion module guided by Laplacian image are adopted, and the feature layer f5 is input into the scale feature interaction module based on collaborative attention. The spatial semantic information of the disc brake image is extracted from the feature layer by the scale feature interaction module based on collaborative attention and the spatial semantic difference is reduced to obtain the feature layer P5. The original input image f1, feature layer f2, feature layer f3, feature layer f4, feature layer f5 and feature layer P5 are input into the cross-scale feature fusion module guided by Laplacian image to obtain the corresponding segmentation mask; The encoder specifically comprises the following steps: first, the input image f1 is processed twice by a downsampling module based on Haar wavelet, and after the first downsampling module is processed based on Haar wavelet, a feature layer f2 is obtained; the feature layer f2 is used as the input of the second downsampling module based on Haar wavelet, and the output of the second downsampling module based on Haar wavelet is sequentially passed through three residual blocks inserted into the global-local feature extraction module, and the output results of the three residual blocks inserted into the global-local feature extraction module are feature layers f3, f4 and f5 respectively; The residual block inserted into the global-local feature extraction module is specifically as follows: a global-local feature extraction module is added after the 1×1 and 3×3 convolutions of the original residual block part, and the global-local feature extraction module performs local adaptive average pooling on the input image to obtain local spatial information; then it is divided into two branches, the first branch performs global adaptive average pooling on the obtained local features, performs a convolution operation when the spatial dimension is reduced to 1, and then expands to the original input size of the module; the second branch adjusts the shape of the obtained local features, extracts deeper features through one-dimensional convolution, and then returns to the original input size of the module through shape adjustment; the features obtained by the two branches are weighted fused, and then restored to the same size as the input features through adaptive inverse average pooling to obtain an attention map; finally, the input features are multiplied by the generated attention map to obtain the output of the global-local feature extraction module.

2. The semantic segmentation method for disc brake vibration displacement measurement according to claim 1, characterized in that: The specific process of the Haar wavelet-based downsampling module is as follows: first, the resolution of the disc brake image is halved; then its signal is decomposed into two parts: a low-frequency L component and high-frequency HH, HV, and HD components, and these are spliced ​​in the channel dimension; finally, a feature map is obtained through convolution, normalization, and ReLU activation function processing.

3. The semantic segmentation method for disc brake vibration displacement measurement according to claim 2, characterized in that: The normalization option uses Group Normalization.

4. The semantic segmentation method for disc brake vibration displacement measurement according to claim 1, characterized in that: The intra-scale feature interaction module based on collaborative attention is specifically as follows: the input feature layer f5 is average pooled, and the average values ​​are taken along the height and width dimensions respectively. The feature maps of the two dimensions are divided into 4 groups of non-overlapping sub-features along the corresponding dimensions; then, a separable one-dimensional convolution with multi-receptive field depth sharing is applied to these sub-features, and they are spliced, normalized and activated respectively, and then adjusted back to the original dimensions to form a spatial attention map in two dimensions; the obtained result is then fused by element-wise multiplication with the initial input feature layer f5 of the intra-scale feature interaction module based on collaborative attention. The weighted features that combine global context dependencies and multi-semantic spatial information are average pooled and normalized on the features, and the dimensions are transformed through two-dimensional convolution to generate query q, key k and value v; then, the self-attention score between k and q is calculated along the channel dimension using the dot product method, and the channel attention relationship weight is obtained through softmax and dot product calculation is performed with v; finally, the output dimension of the self-attention is restored to the input shape, and after average pooling and activation function, the output feature is multiplied with the input weighted feature that integrates global context dependencies and multi-semantic spatial information to obtain the feature layer P5.

5. The semantic segmentation method for disc brake vibration displacement measurement according to claim 1, characterized in that: The Laplacian image-guided cross-scale feature fusion module is specifically: The feature layer f j+1 Perform 2 times upsampling and then combine with the feature layer f j Perform subtraction operations to obtain the Laplacian images L of each scale j ;in, ; Feature layer P5 is fused with feature layer f4 in the first fusion module to obtain output feature layer x4; feature layer P5 is passed through the first convolution module five times to obtain feature layer R5; The feature layer R5 is upsampled twice and then stacked with x4 and L4 to obtain the feature layer P4; the feature layer P4 undergoes four first convolution modules and is added with L4 to obtain R4; P4 undergoes one first convolution module and then passes through the first fusion module to be fused with the feature layer f3 to obtain the output feature layer x3; The feature layer R4 is upsampled twice and then stacked with x3 and L3 to obtain the feature layer P3; the feature layer P3 passes through the first convolution module three times and is added with L3 to obtain R3; P3 passes through the first convolution module once and then passes through the first fusion module once to be fused with the feature layer f2 to obtain the output feature layer x2; The feature layer R3 is upsampled twice and then stacked with x2 and L2 to obtain the feature layer P2; the feature layer P2 passes through the first convolution module twice and is added with L2 to obtain R2; P2 passes through the first convolution module once and then passes through the first fusion module once to be fused with the feature layer f1 to obtain the feature layer x1; The feature layer R2 is upsampled twice and then stacked with x1 and L1 to obtain the feature layer P1; the feature layer P1 is passed through three convolution modules and then added to L1 to obtain R1; R5 is upsampled twice and added to R4 to obtain D4; D4 is upsampled twice and added to R3 to obtain D3; D3 is upsampled twice and added to R2 to obtain D2; D2 is upsampled twice and added to R1 to obtain D1. D1 is the final output result of the semantic segmentation network.

6. A processor, characterized in that: The processor is configured to execute operations including executing the semantic segmentation method for disc brake vibration displacement measurement according to any one of claims 1 to 5.

Images