Bridge expansion joint service state real-time monitoring method and system based on voiceprint feature recognition

Abstract

The invention belongs to the field of bridge monitoring, and discloses a voiceprint feature recognition-based bridge expansion joint service state real-time monitoring method and system. The method comprises the following steps: acquiring original sound pressure signals and video stream data when a vehicle passes through a seam, and constructing sound field tensor data fusing a sound source orientation, multi-channel amplitude-phase characteristics and vehicle parameters; doppler frequency shift correction is completed based on a vehicle speed vector, background noise is stripped by adopting an adaptive wavelet threshold algorithm, and a pure impact sound spectrum feature matrix is generated; mapping the sound spectrum features and the vehicle axle load information to a unified feature space, and generating a fusion feature vector through a multi-head attention mechanism; and inputting a pre-trained deep residual shrinkage network, comparing the pre-trained deep residual shrinkage network with the disease voiceprint reference library, outputting a disease type and confidence, and triggering graded early warning. By accurately capturing, correcting and analyzing the physical acoustic characteristics of the expansion joint, accurate recognition and differential early warning of bridge expansion joint diseases are achieved, and state testing and analysis are completed based on the physical response characteristics of the expansion joint.

Description

Technical Field

[0001] This invention relates to the field of bridge engineering monitoring, and in particular to a method and system for real-time monitoring of the service status of bridge expansion joints based on voiceprint feature recognition. Background Technology

[0002] In the maintenance of transportation structures such as bridges, acoustic testing technology is widely used in the detection of expansion joint defects. This technology collects acoustic signals generated when vehicles pass through expansion joints and extracts defect characteristics using signal processing methods. It enables early monitoring of non-structural defects such as loose bolts and peeling rubber strips. Furthermore, it features non-contact data acquisition and real-time access to on-site inspection information, demonstrating significant effectiveness in the routine maintenance of transportation infrastructure.

[0003] Engineers conducting on-site inspections of expansion joint defects found it difficult to accurately extract effective early-stage acoustic signals from environmental noise. Specifically, when vehicles passed through the expansion joint at different speeds, the collected acoustic signatures showed overlap between defect characteristics and background noise such as tire pump noise and wind noise in the time-frequency domain. Furthermore, the acoustic signature spectrum of the same defect exhibited significant distortion due to changes in vehicle speed. Even after processing with traditional fixed-threshold filtering methods, subtle defect features such as loose bolts and early peeling of rubber strips could not be clearly identified. The root cause of this problem lies in the varying impact excitation intensity generated by different vehicle models and speeds. The Doppler effect caused by high-speed driving alters the acoustic signature spectrum of defects. Simultaneously, the time-frequency characteristics of traffic flow background noise interfere with the defect signals, and traditional filtering methods cannot dynamically adapt to real-time changes in the signal-to-noise ratio. This issue leads to the failure to detect early non-structural defects in expansion joints in a timely manner, allowing the defects to continue to develop. This can result in decreased sealing performance of the expansion joint, uneven structural stress, and subsequent secondary damage such as bridge pavement cracking and beam collisions, increasing the engineering difficulty and economic cost of later repairs. Summary of the Invention

[0004] To overcome the aforementioned deficiencies of the prior art, the present invention provides the following technical solution: A real-time monitoring method for the service status of bridge expansion joints based on acoustic signature recognition includes: collecting raw sound pressure signals and video stream data when vehicles pass through the joint; extracting multi-channel sound field data through a directional microphone array; obtaining vehicle parameters by combining video recognition; constructing sound field tensor data; performing Doppler frequency shift correction based on the vehicle velocity vector in the sound field tensor data; using an adaptive wavelet thresholding algorithm to remove background noise and generate a clean impact acoustic spectrum feature matrix; mapping the clean impact acoustic spectrum feature matrix and vehicle axle load information to a unified feature space; using a multi-head attention mechanism to assign dynamic weights to different frequency bands and outputting a fused feature vector; inputting the fused feature vector into a pre-trained deep residual shrinking network; comparing the topological distance with a defect acoustic signature benchmark database; outputting the defect type and corresponding confidence level; and triggering an early warning strategy.

[0005] Furthermore, the process of constructing sound field tensor data includes: deploying a directional microphone array and acquiring raw sound pressure signals, simultaneously acquiring video stream data, and constructing a raw multimodal acquisition dataset; based on the video stream data in the raw multimodal acquisition dataset, extracting vehicle parameter vectors using vehicle detection and tracking algorithms; based on the eight-channel raw sound pressure sampling sequence in the raw multimodal acquisition dataset, calculating the sound source azimuth sequence using beamforming algorithms; and constructing sound field tensor data based on the eight-channel raw sound pressure sampling sequence, the sound source azimuth sequence, and the vehicle parameter vectors.

[0006] Furthermore, the process for extracting vehicle parameter vectors is as follows: target detection is performed on each frame of the video stream data to identify the vehicle bounding box position and vehicle category in the image; multi-target tracking is performed on the vehicle bounding boxes detected in consecutive frames to establish the association relationship between the same vehicle in different frames and generate a vehicle trajectory sequence; vehicle speed vector is calculated based on the vehicle trajectory sequence; and vehicle axle load information is estimated based on vehicle category labels and vehicle bounding box dimensions.

[0007] Furthermore, the process of generating a pure impact spectral feature matrix includes: extracting motion parameters required for Doppler correction from the sound field tensor data and constructing a Doppler frequency shift compensation coefficient sequence; performing inverse frequency axis correction on the multi-channel sound pressure signals in the sound field tensor data based on the Doppler frequency shift compensation coefficient sequence to obtain a Doppler-corrected sound pressure tensor; performing background noise stripping using an adaptive wavelet thresholding algorithm based on the Doppler-corrected sound pressure tensor to obtain a denoised pressure tensor; and extracting Mel frequency cepstral coefficients based on the denoised pressure tensor to construct a pure impact spectral feature matrix.

[0008] Furthermore, the processing flow of the adaptive wavelet thresholding algorithm is as follows: the Doppler-corrected sound pressure tensor is processed sequentially by channel and time window; for each one-dimensional sound pressure sequence to be processed, a multi-scale discrete wavelet transform is performed to decompose the signal into approximation coefficients and detail coefficients at multiple scales; the noise standard deviation estimate is calculated for the detail coefficients at each scale; the adaptive threshold at each scale is calculated based on the noise standard deviation estimate and the signal length; soft thresholding is performed on the detail coefficients at each scale; and the inverse discrete wavelet transform is performed based on the processed approximation coefficients and detail coefficients at each scale to reconstruct the multi-scale decomposition result into a time-domain signal, thus obtaining the denoised sound pressure sequence.

[0009] Furthermore, the process of outputting the fused feature vector includes: performing frequency band division on the pure impact spectrum feature matrix to construct a multi-frequency band feature sub-matrix set; performing time dimension aggregation on each sub-matrix in the multi-frequency band feature sub-matrix set to obtain a multi-frequency band feature vector set; extracting axle load information from the vehicle parameter vector attached to the pure impact spectrum feature matrix to construct a load-encoded feature vector; mapping the multi-frequency band feature vector set and the load-encoded feature vector to a unified feature space to construct a feature matrix to be fused; and performing multi-head attention mechanism calculation based on the feature matrix to be fused to generate the fused feature vector.

[0010] Further, the calculation process of the multi-head attention mechanism is as follows: Set the number of attention heads, divide the target dimension of the unified feature space by the number of attention heads to obtain the subspace dimension corresponding to each attention head; set a query projection matrix, key projection matrix, and value projection matrix for each attention head, multiply the feature matrix to be fused by the query projection matrix, key projection matrix, and value projection matrix respectively to obtain the query matrix, key matrix, and value matrix corresponding to that attention head; calculate the attention weight matrix for each attention head; calculate the weighted value matrix for each attention head; concatenate the weighted value matrices of all attention heads column-wise to restore the target dimension of the unified feature space, forming the multi-head attention output matrix; perform residual connection and layer normalization processing on the multi-head attention output matrix, perform global average pooling on the layer normalized matrix, compressing the four rows of feature vectors into one row to obtain the fused feature vector.

[0011] Furthermore, the process of outputting the disease type and corresponding confidence level and triggering the early warning strategy includes: constructing a disease voiceprint benchmark library to store standard feature templates for various typical disease states; constructing a deep residual shrinking network and completing offline pre-training for refining the fused feature vectors; inputting the fused feature vectors into the pre-trained deep residual shrinking network to obtain refined feature vectors, and comparing them with the disease voiceprint benchmark library in terms of topological distance; and executing a graded early warning strategy based on the predicted disease type and confidence score.

[0012] Furthermore, the structure and training process of the deep residual shrinkage network are as follows: The hierarchical structure of the deep residual shrinkage network is designed, consisting of an input layer, multiple stacked residual shrinkage modules, and an output layer. The input layer receives the fused feature vector and performs batch normalization. Each residual shrinkage module contains two fully connected layers, a soft thresholding submodule, and an identity skip connection. The output layer maps the output of the last residual shrinkage module to a refined feature vector with the same dimension as the fused feature vector. The internal structure of the soft thresholding submodule is designed. The input of the soft thresholding submodule is the output vector of the two fully connected layers in the residual shrinkage module. First, the mean of the input vector is calculated through a global average pooling layer. Then, the mean is mapped to a threshold ratio between zero and one through two small fully connected layers and an activation function. For example, the threshold coefficient is multiplied by the maximum absolute value of the input vector to obtain the adaptive threshold. Soft thresholding is performed on the input vector, and the output is a sparsed feature vector. Offline pre-training is performed on the deep residual shrinking network using labeled sample data. The fused feature vectors corresponding to the labeled samples of each category are used as input, and the disease category labels of the samples are used as supervision signals. A temporary classification head is added after the output layer to map the refined feature vector to the predicted probability distribution of the five categories. A cross-entropy loss function is defined to measure the difference between the predicted probability distribution and the true label. The stochastic gradient descent algorithm is used to minimize the loss function and update all learnable parameters in the network. Training stops when the classification accuracy on the validation set no longer improves. After training, the temporary classification head is removed, and the network parameters from the input layer to the output layer are retained for online feature refinement.

[0013] A real-time monitoring system for the service status of bridge expansion joints based on voiceprint feature recognition is provided to implement the aforementioned real-time monitoring method for the service status of bridge expansion joints based on voiceprint feature recognition. The system includes: Sound field tensor data construction module: used to collect raw sound pressure signals and video stream data when vehicles pass through gaps, extract multi-channel sound field data through a directional microphone array, combine video recognition to obtain vehicle parameters, and construct sound field tensor data; Pure acoustic spectrum feature generation module: used to perform Doppler frequency shift correction based on vehicle velocity vector in sound field tensor data, and to use an adaptive wavelet threshold algorithm to remove background noise and generate a pure impact acoustic spectrum feature matrix. Fusion Feature Vector Output Module: This module maps the pure impact sound spectrum feature matrix and vehicle axle load information to a unified feature space, uses a multi-head attention mechanism to assign dynamic weights to different frequency bands, and outputs a fusion feature vector. Disease identification and early warning module: It is used to input the fused feature vector into the pre-trained deep residual shrinking network, compare the topological distance with the disease voiceprint benchmark library, output the disease type and corresponding confidence level, and trigger the early warning strategy.

[0014] This invention constructs sound field tensor data through coordinated acquisition of directional microphone arrays and video, overcoming the technical bottleneck of traditional single-channel acoustic monitoring's inability to distinguish the spatial distribution of sound sources. A beamforming algorithm is used to calculate the azimuth sequence of the sound sources, and combined with vehicle parameter vectors to achieve coupling and encapsulation of acoustic spatial information and vehicle dynamics information, effectively improving the accuracy of abnormal sound source localization and achieving effective separation of acoustic responses at different locations of expansion joints. Doppler frequency shift compensation coefficients are calculated using vehicle velocity vectors, and frequency axis inverse correction is performed on multi-channel sound pressure signals, unifying the acoustic signature features at different vehicle speeds to a stationary reference frame and eliminating spectral distortion caused by vehicle speed changes. Background noise is stripped using an adaptive wavelet threshold algorithm. Through multi-scale noise estimation and dynamic threshold adjustment, the signal-to-noise ratio is effectively improved while preserving impact signal details, solving the interference problem of overlapping environmental noise and impact acoustic signature frequency domains. A multi-head attention mechanism is introduced to dynamically weight multi-band features, and load normalization processing is achieved by combining axle load information, effectively reducing the difference in defect features under light and heavy vehicle excitation and avoiding interference from differences in excitation intensity on the identification results. Ultimately, by comparing the topological distance between the deep residual shrinkage network and the defect acoustic signature benchmark library, the accurate classification of five types of defects, such as concrete breakage in the anchorage zone and fracture of the middle beam, was achieved. This provides a quantitative analysis method for dynamic characteristic testing and health status assessment of key structural components such as bridge expansion joints, and meets the high-precision requirements for structural component condition monitoring in the field of static or dynamic balance testing of machines or structural components. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart of the real-time monitoring method for the service status of bridge expansion joints based on voiceprint feature recognition in this invention. Figure 2 This is a schematic diagram illustrating the deployment and collaborative monitoring of the directional microphone array in an embodiment of the present invention; Figure 3 This is a schematic diagram of beamforming sound source azimuth angle scanning in an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the calculation of the radial velocity component in an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the soft threshold processing principle in an embodiment of the present invention; Figure 6 This is a schematic diagram illustrating the construction of load encoding feature vectors in an embodiment of the present invention; Figure 7 This is a schematic diagram of graded early warning triggering in an embodiment of the present invention; Figure 8 This is a functional block diagram of the bridge expansion joint service status real-time monitoring system based on voiceprint feature recognition in this invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example 1: Please see Figure 1 As shown, this embodiment provides a method for real-time monitoring of the service status of bridge expansion joints based on voiceprint feature recognition, including: S1: Collect raw sound pressure signals and video stream data when the vehicle passes through the gap, extract multi-channel sound field data through a directional microphone array, and combine video recognition to obtain vehicle parameters to construct sound field tensor data.

[0019] This step addresses the problem that single-channel acquisition cannot capture the spatial distribution characteristics of sound sources in the acoustic monitoring of bridge expansion joints. Based on the collaborative work of a directional microphone array and a video acquisition system, it constructs sound field tensor data that comprehensively reflects the sound source's location, multi-channel amplitude and phase characteristics, and vehicle dynamic parameters. This sound field tensor data, as the final output of S1, will be used in S2 for Doppler frequency shift correction and background noise stripping.

[0020] Specifically, the process of constructing the sound field tensor data includes: S11: Deploy a directional microphone array and collect raw sound pressure signals, simultaneously collect video stream data, and construct a raw multimodal acquisition data set.

[0021] In this step, the bridge expansion joint monitoring section is the core area. A uniform linear array consisting of eight omnidirectional electret microphones is deployed symmetrically on both sides of the expansion joint. The spacing between adjacent microphones in the array is determined according to the wavelength corresponding to the upper limit of the target monitoring frequency range to meet the requirements of the spatial sampling theorem. The directional microphone array is connected to the edge computing unit through a multi-channel synchronous acquisition card. The sampling frequency of the acquisition card is determined according to the highest frequency component of the impact sound pattern of the expansion joint, and is usually set to more than 2.5 times the highest target frequency to avoid spectral aliasing. At the same time, an industrial camera is deployed at a fixed position upstream of the expansion joint, with a field of view covering the entire lane width before the vehicle enters the monitoring area. The video frame rate is set to more than 25 frames per second to ensure the continuity of the vehicle trajectory.

[0022] The raw sound pressure signal refers to the time-domain voltage sequence output by each microphone unit in the microphone array during the vehicle's passage through the expansion joint, which is stored as a discrete sampling point sequence after pre-amplification and analog-to-digital conversion. The video stream data refers to the continuous image frame sequence output by the industrial camera, with each frame containing the vehicle's position information and appearance features within the monitoring area. The raw multimodal acquisition data set is a data set formed by timestamping the raw sound pressure signal and video stream data according to the unified clock of the edge computing unit, including an eight-channel raw sound pressure sampling sequence, a video image frame sequence, and the corresponding timestamp sequence. See also Figure 2This figure illustrates a directional microphone array deployment and collaborative monitoring method provided in this application. As shown, a uniform linear array consisting of eight omnidirectional electret microphones is deployed upstream of the bridge expansion joint, mounted on dedicated brackets near both sides of the expansion joint to ensure effective spatial sampling of the impact sound field. An industrial camera is fixedly installed directly above the upstream lane of the expansion joint. Its field of view (orange dashed line and filled area in the figure) completely covers the entire lane width before a vehicle enters the monitoring area, forming a rectangular monitoring area (blue dashed box) defined by the camera and microphone array. In this figure, a vehicle is passing through this area in the designated direction. The microphone array is connected to an edge computing unit (not shown in the figure) via a multi-channel synchronous acquisition card. In the acoustic monitoring scenario of the bridge expansion joint, the location of defects (such as anchorage areas, central beams, and rubber strips) has clear spatial attributes. The core value of deploying a multi-channel microphone array instead of a single-point microphone lies in its ability to estimate the spatial azimuth of abnormal sound sources through array signal processing technologies such as beamforming. This allows for the spatial separation and localization of acoustic responses from different structural parts of the expansion joint, which is crucial for distinguishing between defects occurring in different locations, such as "concrete breakage in the anchorage zone" and "fracture of the central beam." Simultaneously, the introduction of a video acquisition system, synchronized spatiotemporally with the acoustic array, enables precise acquisition of vehicle parameters (such as vehicle type, number of axles, and speed) that trigger acoustic events. This provides key input for subsequent steps to eliminate acoustic characteristic variations caused by differences in excitation sources (vehicle load, speed), achieving an upgrade from a "passive sound recording" to an "active association of sound sources and excitations" monitoring mode.

[0023] S12: Based on the video stream data in the original multimodal acquisition dataset, vehicle parameter vectors are extracted using vehicle detection and tracking algorithms.

[0024] Specifically, the process for extracting the vehicle parameter vector is as follows: S121: Perform object detection on each frame of the video stream data to identify the location of vehicle bounding boxes and vehicle categories in the images. The object detection uses a pre-trained convolutional neural network model trained on a large-scale traffic scene dataset. This model outputs the top-left corner coordinates, width, height of the vehicle bounding boxes, and vehicle category labels. The vehicle category labels include five categories: small passenger vehicles, medium-sized passenger vehicles, large trucks, heavy trucks, and special vehicles. Different categories correspond to different axle count ranges and axle load estimation intervals.

[0025] S122: Perform multi-target tracking on the vehicle bounding boxes detected in consecutive frames, establish the association between the same vehicle in different frames, and generate a vehicle trajectory sequence. The multi-target tracking adopts an inter-frame matching strategy based on the Hungarian algorithm, using a weighted combination of the Euclidean distance between the center points of the vehicle bounding boxes in adjacent frames and the cosine similarity of appearance features as the matching cost. The weight coefficients are adaptively adjusted according to the vehicle density and occlusion degree in the tracking scene. Each vehicle trajectory sequence contains the bounding box center coordinates of the vehicle in each frame and the corresponding timestamp.

[0026] S123: Calculate the vehicle velocity vector based on the vehicle trajectory sequence. The calculation method for the vehicle velocity vector is as follows: Select the center coordinates of the bounding box corresponding to adjacent timestamps in the vehicle trajectory sequence, calculate the coordinate difference and divide it by the timestamp difference to obtain the velocity component in the pixel coordinate system; through the pre-calibrated camera intrinsic parameter matrix and road surface homography matrix, convert the velocity component in the pixel coordinate system into a velocity vector in the actual physical coordinate system, with the unit being meters per second. The velocity vector includes a longitudinal velocity component along the lane direction and a lateral velocity component perpendicular to the lane direction.

[0027] S124: Estimate vehicle axle load information based on vehicle category labels and vehicle bounding box dimensions. The axle load estimation method is as follows: Establish a mapping table between vehicle categories and typical axle numbers, where small passenger cars correspond to two axles, medium-sized passenger cars correspond to two axles, large trucks correspond to three axles, heavy trucks correspond to four to six axles, and special vehicles correspond to six or more axles; based on the ratio of the projected area of ​​the vehicle bounding box in the image to the projected area of ​​the standard vehicle model, combined with the unloaded axle load and fully loaded axle load range corresponding to the vehicle category, estimate the average axle load value per axle of the current vehicle through linear interpolation. The vehicle parameter vector is a one-dimensional numerical vector formed by arranging the vehicle category label, axle number, speed vector, and average axle load value per axle in a fixed order, with one vehicle parameter vector corresponding to each vehicle.

[0028] S13: Based on the eight-channel original sound pressure sampling sequence in the original multimodal acquisition data set, the sound source azimuth sequence is calculated using a beamforming algorithm.

[0029] Specifically, the processing flow of the beamforming algorithm is as follows: S131: Perform time-window framing processing on the eight-channel raw sound pressure sampling sequence. The length of the time window is determined based on the typical duration of the impact sound pattern of the expansion joint, and is usually set to 50 to 100 milliseconds; the overlap rate between adjacent time windows is set to 50% to ensure the continuity of time resolution. Each time window contains a set of synchronous sampling points for eight channels, forming a time-window sound pressure matrix.

[0030] S132: Perform a frequency domain transformation on the sound pressure matrix of each time window to obtain the time window frequency domain matrix. The frequency domain transformation uses the Fast Fourier Transform algorithm to convert the time domain sampling point sequence into a complex frequency domain coefficient sequence, retaining the positive frequency part for subsequent processing.

[0031] S133: Based on the time window frequency domain matrix and microphone array geometric parameters, a delay-summation beamforming method is used to scan a preset azimuth range and calculate the beam power output value corresponding to each azimuth angle. The preset azimuth range uses the normal to the centerline of the expansion joint as the zero-degree reference and covers a spatial range from -90 degrees to +90 degrees, with a scanning step angle set to one degree. For each scanning azimuth angle, based on the time delay difference of the plane wave arriving at each microphone corresponding to that azimuth angle, phase compensation is applied to the frequency domain coefficients of each channel in the time window frequency domain matrix. The compensated frequency domain coefficients of each channel are then summed using complex numbers, and the square of the modulus is calculated to obtain the beam power output value corresponding to that azimuth angle. (See also...) Figure 3 This is a schematic diagram of beamforming sound source azimuth scanning provided in an embodiment of this application. As shown in the figure, with the center of the microphone array located at the bottom of the figure as the origin and the normal of the expansion joint centerline as the zero-degree reference, the spatial range of -90 degrees to +90 degrees is scanned in front of the array (i.e., the space on the bridge). The figure uses dashed lines to indicate the scanning direction lines every 15 degrees. One of the solid arrows located at approximately +30 degrees represents the direction of the peak beam power calculated within the scanning time window, i.e., the estimated azimuth of the main sound source, and is marked with a star icon. The essence of the beamforming algorithm is to perform delay compensation and coherent superposition on the signals received by each channel of the array, and to "listen" to the signal energy from that direction by scanning different azimuths. In the complex sound field environment of bridge expansion joint monitoring, the sound signal is a mixture of various components such as vehicle tire noise, expansion joint impact sound, and environmental background noise. This step, by performing beamforming scanning and extracting the peak azimuth angle by time window, can dynamically track the source direction of the main component of the impact sound. When expansion joints experience localized damage (e.g., damage to one side of the anchorage zone), the spatial orientation of the resulting abnormal impact sound will differ identibly from the orientation of normal traffic noise or the symmetrical impact sound from a healthy expansion joint. This spatial orientation information is appended to the sound field tensor as crucial metadata, providing important clues for subsequent analysis regarding "where the sound source is," enhancing the system's ability to perceive the spatial location of the damage, and preventing the confusion of acoustic events from different directions.

[0032] S134: Search for the peak position in the beam power output value sequence of each time window, and use the azimuth angle corresponding to the peak as the source azimuth angle estimate for that time window. If there are multiple significant peaks within the same time window, it indicates the existence of multiple sound sources. Record the azimuth angles corresponding to the first three peaks in descending order of peak power. Arrange the source azimuth angle estimates of all time windows in chronological order to form a source azimuth angle sequence.

[0033] S14: Construct sound field tensor data based on the eight-channel original sound pressure sampling sequence, sound source azimuth sequence, and vehicle parameter vector.

[0034] Specifically, the construction process of the sound field tensor data is as follows: S141: Perform amplitude normalization processing on the eight-channel raw sound pressure sampling sequence. The amplitude normalization method is as follows: calculate the root mean square value of the sound pressure sampling values ​​of each channel throughout the entire acquisition period, divide all sampling values ​​of that channel by the root mean square value, and obtain the normalized sound pressure sampling sequence. Normalization processing eliminates the influence of differences in the sensitivity of different microphones on subsequent feature extraction.

[0035] S142: The normalized sound pressure sampling sequence is recombined according to the time window framing results. The normalized sampling points of the eight channels in each time window are arranged into a two-dimensional matrix, with the row index corresponding to the channel number and the column index corresponding to the sampling point number in the window, forming a time window multi-channel sound pressure matrix.

[0036] S143: Associate the multi-channel sound pressure matrix of the time window with the estimated azimuth angle of the sound source for the corresponding time window, and attach an azimuth angle metadata tag to the multi-channel sound pressure matrix of each time window. When a vehicle passing through an expansion joint is detected within a certain time window, the time window is marked as an effective impact time window, and the corresponding vehicle parameter vector is attached to the metadata of that time window.

[0037] S144: Stack the multi-channel sound pressure matrices of all effective impact time windows in chronological order to form a three-dimensional tensor structure. The first dimension corresponds to the time window index, the second dimension to the channel index, and the third dimension to the sampling point index within the window. The sound source azimuth sequence and vehicle parameter vector are encapsulated as attribute fields of the tensor, ultimately forming the sound field tensor data. This sound field tensor data is a composite data object containing a three-dimensional numerical tensor ontology and structured metadata. The metadata fields include the acquisition timestamp, sound source azimuth sequence, vehicle type, number of vehicle axles, vehicle velocity vector, and average axle load per axle.

[0038] Specifically, this step employs a directional microphone array for multi-channel sound pressure acquisition, rather than a single-channel recording device, because the location of defects at bridge expansion joints exhibits spatial distribution characteristics. When concrete in the anchorage zone breaks or the central beam fractures, the spatial location of the abnormal sound source differs from that of normal traffic noise. Single-channel acquisition cannot distinguish this spatial difference, causing the defect's acoustic signature to be submerged in background noise and difficult to identify. By acquiring sound through an eight-channel array and performing beamforming processing, the spatial azimuth angle of the sound source can be estimated, thereby separating sounds from different parts of the expansion joint in subsequent processing. Simultaneously, a video acquisition system is introduced to obtain vehicle parameter vectors because the impact excitation intensity generated by vehicles of different models and loads passing through the expansion joint varies significantly. If the characteristics of the excitation source are not calibrated, the acoustic signature differences exhibited by the same defect under light-load and heavy-load vehicle excitation may be misjudged as different health states. Using the axle load information in the vehicle parameter vector, load normalization processing can be performed on the acoustic characteristics in subsequent steps, eliminating the interference of excitation differences on defect identification. The acoustic field tensor data constructed in this step couples and encapsulates acoustic spatial information with vehicle dynamics information, enabling downstream processing modules to utilize both types of information simultaneously. This achieves an upgrade in acoustic monitoring from scalar acquisition to high-dimensional tensor acquisition, providing a complete input data foundation for subsequent Doppler correction and load normalization.

[0039] S2: Doppler frequency shift correction is performed based on the vehicle velocity vector in the sound field tensor data, and an adaptive wavelet threshold algorithm is used to remove background noise and generate a clean impact sound spectrum feature matrix.

[0040] This step addresses the issue of the Doppler effect caused by vehicle motion, which results in the same defect exhibiting different spectral characteristics at different vehicle speeds. Based on the vehicle velocity vector and multi-channel sound pressure information encapsulated in the sound field tensor data, a clean impact sound spectrum feature matrix is ​​generated after eliminating the influence of speed and background noise through inverse Doppler correction and adaptive wavelet denoising. This clean impact sound spectrum feature matrix, as the final output of S2, will be used in S3 for feature fusion with vehicle axle load information.

[0041] Specifically, the process for generating the pure impact spectral feature matrix includes: S21: Extract the motion parameters required for Doppler correction from the sound field tensor data and construct the Doppler frequency shift compensation coefficient sequence.

[0042] Specifically, the construction process of the Doppler frequency shift compensation coefficient sequence is as follows: S211: Read the vehicle velocity vector and sound source azimuth sequence from the metadata field of the sound field tensor data. The vehicle velocity vector contains longitudinal and lateral velocity components, which are vector-synthesized to obtain the magnitude and azimuth angle of the synthesized vehicle velocity. The sound source azimuth sequence contains the estimated sound source azimuth angle for each effective impact time window.

[0043] S212: Calculate the radial velocity component of the sound source relative to the microphone array within each effective impact time window. The radial velocity component is calculated as follows: The difference between the direction angle of the vehicle's synthesized velocity and the azimuth angle of the sound source within that time window is calculated to obtain the angle between the velocity vector and the line connecting the sound source to the array; the magnitude of the vehicle's synthesized velocity is multiplied by the cosine of this angle to obtain the radial velocity component. The radial velocity component is positive when the vehicle is moving towards the array and negative when the vehicle is moving away from the array. See also... Figure 4 This is a schematic diagram of radial velocity component calculation provided in an embodiment of this application. As shown in the figure, the schematic diagram is a top view showing a moving vehicle (simplified as a truck outline on the left), a fixed microphone array (rectangular area on the right), and the geometric relationship between the three. The black solid arrow represents the vehicle velocity vector (i.e., the composite velocity of the vehicle on the ground), and the dashed line represents the line connecting the sound source to the array (i.e., the direction of the sound wave propagation path). The red thick solid arrow represents the radial velocity component calculated by projecting the vehicle velocity vector onto the line connecting the sound source and the array. Radial velocity is a direct physical quantity that causes Doppler frequency shift; its magnitude is the projected length of the vehicle velocity vector in the direction of sound wave propagation, and its direction determines whether the frequency increases (approaching) or decreases (moving away). In a monitoring scenario where a moving vehicle passes through a fixed microphone array, if the Doppler effect is not considered, the characteristic frequencies generated by the same expansion joint defect will undergo linear shifts due to different vehicle speeds, leading to feature inaccuracies. This step, by combining the vehicle velocity vector obtained in S1 (from video analysis) and the sound source azimuth sequence (from beamforming), accurately calculates the radial velocity of the sound source relative to the array within each time window, providing key parameters for subsequent frequency axis inverse correction. This calculation transforms the Doppler problem of moving sound sources into a deterministic geometric correction problem based on measured motion parameters. This allows the acoustic spectra collected at different vehicle speeds to be uniformly restored to the "stationary sound source" reference frame, ensuring the comparability of acoustic features across speeds. This is a crucial preliminary step for achieving highly robust defect identification.

[0044] S213: Calculate the Doppler frequency shift scaling factor based on the radial velocity component and the sound speed constant. The sound speed constant is taken as the sound speed in air under standard atmospheric pressure. The Doppler frequency shift scaling factor is calculated by dividing the sound speed constant by the quotient of the difference between the sound speed constant and the radial velocity component, thus obtaining the Doppler frequency shift scaling factor corresponding to the time window. Arrange the Doppler frequency shift scaling factors of all effective impact time windows in chronological order to form a Doppler frequency shift compensation coefficient sequence.

[0045] S22: Perform frequency axis inverse correction on the multi-channel sound pressure signal in the sound field tensor data based on the Doppler frequency shift compensation coefficient sequence to obtain the Doppler-corrected sound pressure tensor.

[0046] Specifically, the frequency axis inverse correction process is as follows: S221: Extract the three-dimensional numerical tensor ontology from the sound field tensor data and process it window by window according to the time window index. For each time window, extract the eight-channel normalized sound pressure matrix corresponding to that window.

[0047] S222: Perform a Fast Fourier Transform on each channel of the eight-channel normalized sound pressure matrix to obtain the complex spectrum vector of that channel. Each element of the complex spectrum vector corresponds to a discrete frequency point, containing the amplitude and phase information of that frequency point.

[0048] S223: Read the Doppler frequency shift scaling factor corresponding to the current time window from the Doppler frequency shift compensation coefficient sequence, and divide the frequency axis scale of the complex spectrum vector by this scaling factor to achieve frequency axis scaling transformation. The frequency axis scaling transformation adopts a linear interpolation method: for each target frequency point on the transformed frequency axis, find the two nearest source frequency points on the original frequency axis, and perform linear interpolation on the complex spectrum values ​​corresponding to these two source frequency points to obtain the corrected complex spectrum value of the target frequency point.

[0049] S224: Perform an inverse fast Fourier transform on the corrected complex spectral vector to restore the frequency domain signal to the time domain signal, obtaining the Doppler-corrected sound pressure sequence for that channel. Recombine the Doppler-corrected sound pressure sequences of the eight channels into a Doppler-corrected sound pressure matrix.

[0050] S225: Re-stack the Doppler-corrected sound pressure matrices of all time windows according to the original time window index to form a Doppler-corrected sound pressure tensor. The dimensional structure of the Doppler-corrected sound pressure tensor is the same as that of the three-dimensional numerical tensor in the sound field tensor data, but the Doppler frequency shift distortion caused by vehicle motion has been eliminated for each frequency component.

[0051] S23: Based on the sound pressure tensor after Doppler correction, an adaptive wavelet threshold algorithm is used to perform background noise stripping to obtain the denoised pressure tensor.

[0052] Specifically, the processing flow of the adaptive wavelet thresholding algorithm is as follows: S231: The Doppler-corrected sound pressure tensor is processed sequentially by channel and time window. For each one-dimensional sound pressure sequence to be processed, a multi-scale discrete wavelet transform is performed to decompose the signal into approximation coefficients and detail coefficients at multiple scales. The wavelet basis function used in the discrete wavelet transform is an orthogonal wavelet with compact support characteristics. The number of decomposition levels is determined according to the signal length and target frequency resolution, and is usually set to four to six levels.

[0053] S232: Calculate the noise standard deviation estimate for the detail coefficients at each scale. The noise standard deviation estimate is calculated as follows: take the median of the absolute values ​​of the detail coefficients at that scale, divide the median by the coefficient corresponding to the third quartile of the standard normal distribution, and obtain the noise standard deviation estimate for that scale. This method is robust to outliers and can accurately estimate the background noise level even in the presence of impact signals.

[0054] S233: Calculate the adaptive threshold for each scale based on the noise standard deviation estimate and the signal length. The adaptive threshold is calculated by multiplying the noise standard deviation estimate by the square root of the natural logarithm of the signal length, and then multiplying by a scale-dependent adjustment factor to obtain the adaptive threshold for that scale. The scale-dependent adjustment factor is determined based on the prior energy distribution ratio of the target signal and noise within the corresponding frequency range of each scale. For low-frequency scales where tire pump noise is concentrated, the adjustment factor is appropriately increased to enhance the suppression effect; for mid-to-high-frequency scales where impact signal energy is concentrated, the adjustment factor is appropriately decreased to preserve signal details.

[0055] S234: Perform soft thresholding on the detail coefficients at each scale. The soft thresholding method is as follows: for each coefficient value in the detail coefficient sequence, if its absolute value is less than or equal to the adaptive threshold for that scale, the coefficient is set to zero; if its absolute value is greater than the adaptive threshold for that scale, the sign of the coefficient remains unchanged, and its absolute value is subtracted from the adaptive threshold to obtain the new absolute value. Soft thresholding can reduce ringing effects in signal reconstruction while suppressing noise. See also Figure 5This is a schematic diagram illustrating the soft thresholding principle provided in this application embodiment. As shown in the figure, the horizontal axis of the coordinate system represents the sequence number of detail coefficients at a certain scale after wavelet transform, and the vertical axis represents the coefficient value. The black vertical line segments represent the original detail coefficients, where coefficients exceeding the positive and negative thresholds (two gray dashed lines) are considered to potentially contain effective impact signal components, while coefficients within the threshold are considered to be mainly background noise. The red arrows illustrate the soft thresholding process: for coefficients exceeding the threshold, their absolute value is reduced by the threshold size (shrinking towards zero), while the sign remains unchanged; coefficients not exceeding the threshold are set to zero. The legend distinguishes between coefficients to be "set to zero" (gray line segments) and coefficients to be "shrinked" (black line segments). In the acoustic monitoring of bridge operation, background noise (such as tire pump noise and wind noise) and target impact signals are highly mixed in the time and frequency domain. The core advantage of adaptive wavelet thresholding denoising lies in its multi-scale characteristics: it independently estimates the noise level and sets the threshold at different frequency sub-bands (scales). For the low-frequency subband dominated by pumping noise, a higher threshold can be set for strong suppression; for the mid-to-high-frequency subband where impact signals may converge, a lower threshold is set to preserve signal details. The "soft thresholding" method shown in the figure, while setting the noise figure to zero, avoids signal distortion or ringing effects that may be caused by hard thresholding by smoothly shrinking the signal coefficient. This processing can extract the pure acoustic response related to the physical process of expansion joint impact from the original signal with extremely low signal-to-noise ratio, significantly improving the stability of subsequent feature extraction and the significance of disease-related features.

[0056] S235: Based on the processed approximation coefficients and detail coefficients at each scale, perform an inverse discrete wavelet transform to reconstruct the multi-scale decomposition results into a time-domain signal, obtaining a denoised sound pressure sequence. Recombine the denoised sound pressure sequences of all channels and time windows to form a denoised pressure tensor.

[0057] S24: Extract Mel frequency cepstral coefficients based on the denoised compression tensor to construct a pure impact spectral feature matrix.

[0058] Specifically, the Mel frequency cepstral coefficient extraction process is as follows: S241: The denoised pressure tensor is processed according to time windows. The eight-channel denoised sound pressure sequences within each time window are summed using channel weighting to obtain a single-channel synthesized sound pressure sequence. The weights of the channel weighting are determined based on the angular difference between the azimuth angle of the sound source corresponding to each channel and the azimuth angle of the expansion joint centerline. The smaller the angular difference, the greater the weight of the channel, in order to enhance the sound signal from the direction of the expansion joint and suppress interference signals from other directions. The weights are normalized before the weighted summation operation.

[0059] S242: Perform pre-emphasis processing on the single-channel synthesized sound pressure sequence to enhance the energy proportion of high-frequency components. The pre-emphasis processing uses a first-order high-pass filter, and the filter coefficients are set according to the typical pre-emphasis coefficients in speech signal processing.

[0060] S243: Perform framing and windowing processing on the pre-emphasized signal. The frame length is set to 20 to 40 milliseconds, and the frame shift is set to half the frame length. The windowing uses a Hamming window function to reduce spectral leakage.

[0061] S244: Perform a Fast Fourier Transform on the windowed signal for each frame to calculate the power spectral density. The power spectral density is the square of the complex spectral magnitude divided by the frame length.

[0062] S245: Construct a Mel filter bank and filter the power spectral density. The Mel filter bank consists of several triangular filters, with the center frequencies of each filter evenly distributed on the Mel frequency scale. The number of filters is set according to the target feature dimension, typically between twenty-six and forty. The power spectral density is multiplied point-by-point by each triangular filter and summed to obtain the output energy value of the filter bank corresponding to that filter.

[0063] S246: Take the natural logarithm of the output energy value of the filter bank, then perform a discrete cosine transform, and take the first twelve to thirteen coefficients of the discrete cosine transform result as the Mel frequency cepstral coefficient vector of the frame.

[0064] S247: Arrange the Mel frequency cepstral coefficient vectors of all frames within the effective impact time window according to the time frame index and coefficient index to form a two-dimensional matrix structure. The row index of the two-dimensional matrix corresponds to the time frame number, and the column index corresponds to the order of the Mel frequency cepstral coefficients. This two-dimensional matrix is ​​the pure impact acoustic spectrum feature matrix. At the same time, the vehicle parameter vector in the sound field tensor data is appended as metadata to the pure impact acoustic spectrum feature matrix so that the axle load information can be used for feature fusion in subsequent steps.

[0065] Specifically, the necessity of performing Doppler frequency shift correction in this step lies in the fact that when a vehicle passes through an expansion joint at different speeds, the frequency components of the impact sound pattern will undergo varying degrees of frequency shift. For high-speed vehicles, without Doppler correction, the characteristic frequencies generated by the same type of defect may shift to frequency bands similar to those of another type of defect, leading to confusion in the classification model. By calculating an accurate Doppler frequency shift scaling factor based on the vehicle velocity vector and the sound source azimuth angle and performing inverse correction, the sound patterns collected under different vehicle speed conditions can be uniformly restored to the standard spectrum in a stationary reference frame, making the subsequent feature extraction results comparable across operating conditions. The reason for using an adaptive wavelet threshold algorithm for background noise stripping is that there are pumping noise, wind noise, and other environmental noises generated by tire-road friction in the bridge operating environment. These noise components overlap with the impact sound patterns of the expansion joint in the frequency domain, and traditional fixed threshold denoising methods cannot adapt to the dynamic changes in noise levels. The adaptive wavelet thresholding algorithm estimates the noise standard deviation by scale and sets a scale-dependent threshold. It can suppress background noise in each frequency band while preserving the effective components of the impact signal. This allows the generated pure impact acoustic spectrum feature matrix to accurately reflect the acoustic response characteristics of the expansion joint rather than the environmental noise characteristics, providing a high signal-to-noise ratio feature input for subsequent disease identification.

[0066] S3: Map the pure impact sound spectrum feature matrix and vehicle axle load information to a unified feature space, and use a multi-head attention mechanism to assign dynamic weights to different frequency bands, outputting a fused feature vector.

[0067] This step addresses the issue of different vehicle models and loads affecting acoustic response, leading to the masking of defect characteristics by variations in excitation intensity. Based on the pure impact acoustic spectrum feature matrix and axle load information from the additional vehicle parameter vector, a fused feature vector is generated to eliminate the influence of excitation differences through feature space mapping and a multi-head attention weighting mechanism. This fused feature vector, as the final output of S3, will be used in S4 for defect type classification and confidence calculation.

[0068] Specifically, the process of outputting the fused feature vector includes: S31: Perform frequency band division on the pure impact sound spectrum feature matrix to construct a set of multi-band feature sub-matrices.

[0069] Specifically, the frequency band allocation process is as follows: S311: Based on the acoustic response frequency characteristics of various defects in expansion joints, the frequency range corresponding to the Mel frequency cepstral coefficients is divided into three sub-intervals: low-frequency, mid-frequency, and high-frequency. The low-frequency band corresponds to the first to fourth orders of the Mel frequency cepstral coefficients, mainly reflecting the aging of rubber strips and the low-frequency vibration characteristics of the overall structure; the mid-frequency band corresponds to the fifth to eighth orders of the Mel frequency cepstral coefficients, mainly reflecting the overall stiffness changes and medium-scale damage characteristics of the expansion joint; the high-frequency band corresponds to the ninth to twelfth orders of the Mel frequency cepstral coefficients, mainly reflecting the local high-frequency response characteristics such as metal component impact, bolt loosening, and microcracks.

[0070] S312: The pure impact sound spectrum feature matrix is ​​segmented by column index. The columns corresponding to the first to fourth orders are extracted as low-frequency feature sub-matrices, the columns corresponding to the fifth to eighth orders are extracted as mid-frequency feature sub-matrices, and the columns corresponding to the ninth to twelfth orders are extracted as high-frequency feature sub-matrices. The multi-frequency feature sub-matrices set contains three elements: low-frequency feature sub-matrices, mid-frequency feature sub-matrices, and high-frequency feature sub-matrices.

[0071] S32: Perform time-dimensional aggregation on each submatrix in the multi-band feature submatrix set to obtain a multi-band feature vector set.

[0072] Specifically, the processing flow for the time dimension aggregation is as follows: S321: Perform row-wise statistical operations on the low-frequency band feature submatrix, mid-frequency band feature submatrix, and high-frequency band feature submatrix respectively. The row-wise statistical operations include calculating two statistical measures for each column: the mean and the standard deviation, which reflect the average level and fluctuation degree of the frequency band characteristics throughout the entire impact process.

[0073] S322: Concatenate the mean and standard deviation of each column of each sub-matrix in order to form the aggregated feature vector for that frequency band. The low-frequency band feature sub-matrix is ​​aggregated to form a low-frequency band aggregated feature vector, the mid-frequency band feature sub-matrix is ​​aggregated to form a mid-frequency band aggregated feature vector, and the high-frequency band feature sub-matrix is ​​aggregated to form a high-frequency band aggregated feature vector. The multi-band feature vector set contains three elements: the low-frequency band aggregated feature vector, the mid-frequency band aggregated feature vector, and the high-frequency band aggregated feature vector.

[0074] S33: Extract axle load information from the vehicle parameter vector attached to the pure impact sound spectrum feature matrix and construct a load coding feature vector.

[0075] Specifically, the construction process of the load encoding feature vector is as follows: S331: Read three fields from the vehicle parameter vector: vehicle category label, number of axles, and average axle load per axle. See also Figure 6This is a schematic diagram illustrating the construction of a load-encoded feature vector according to an embodiment of this application. As shown in the figure, the entire process, from left to right, demonstrates how the monitored vehicle information (simplified to a side view of a truck on the left) is encoded into a fixed-length numerical vector. The process consists of three steps: First, vehicle category identification is performed through video analysis, outputting a 5-dimensional one-hot code (in the example in the figure, the second position is 1, representing "large truck"); second, the number of axles is extracted and normalized based on the vehicle type and bounding box information; third, the average axle load per axle is estimated and normalized based on the vehicle type and projected area. Finally, these three parts are concatenated into a 7-dimensional load-encoded feature vector (5-dimensional category code + 1-dimensional number of axles + 1-dimensional axle load). In expansion joint health monitoring, the vehicle is the excitation source, and its load (axle load) and structure (number of axles, vehicle type) directly determine the impact energy and spectral characteristics applied to the expansion joint. When a heavy-duty truck passes through, even if the expansion joint is healthy, the amplitude of its acoustic response may be much higher than that of a light-duty passenger car passing through a damaged expansion joint. Without distinguishing the excitation source, the absolute value of acoustic features loses its evaluative meaning. The core purpose of constructing the load-encoded feature vector in this step is to provide a quantitative description of the "excitation source intensity" for the subsequent fusion model. By inputting this information along with the acoustic features into the model, the system can learn the intrinsic relationship between acoustic response and excitation intensity, thereby achieving "implicit normalization" of acoustic features. This allows the model to focus on the "structural response characteristics under unit excitation" rather than the "absolute magnitude of the response," fundamentally solving the problem of false alarms and false negatives caused by the randomness of traffic flow composition and improving the generalization ability of the defect identification model to complex traffic loads.

[0076] S332: Perform one-hot encoding transformation on the vehicle category label. The one-hot encoding transformation method is as follows: create a zero vector of length five, and set the element at the corresponding position of the vehicle category label to one. Small passenger vehicles correspond to the first position, medium passenger vehicles to the second position, large trucks to the third position, heavy trucks to the fourth position, and special vehicles to the fifth position.

[0077] S333: Perform interval normalization on the number of axles and the average axle weight per axle. The interval normalization method is as follows: based on the minimum number of axles, maximum number of axles, minimum axle weight, and maximum axle weight obtained from historical data collection, subtract the minimum number of axles from the current number of axles and divide by the difference between the maximum and minimum number of axles to obtain the normalized number of axles; subtract the minimum axle weight from the current average axle weight per axle and divide by the difference between the maximum and minimum axle weight to obtain the normalized axle weight.

[0078] S334: The vehicle category vector, normalized axle count, and normalized axle weight, obtained through one-hot encoding, are concatenated in sequence to form a load encoding feature vector. The load encoding feature vector has a length of seven and includes five-dimensional category encoding and two-dimensional numerical encoding.

[0079] S34: Map the multi-band feature vector set and the load-coded feature vector to a unified feature space to construct the feature matrix to be fused.

[0080] Specifically, the feature space mapping process is as follows: S341: Set the target dimension of the unified feature space. The target dimension is determined based on the balance between the computational efficiency of the subsequent attention mechanism and the feature representation capability. It is usually set to sixty-four or one hundred and twenty-eight dimensions.

[0081] S342: Perform linear projection transformations on the low-frequency aggregated feature vector, mid-frequency aggregated feature vector, high-frequency aggregated feature vector, and load coding feature vector, respectively. The linear projection transformation method is as follows: set a projection weight matrix for each source feature vector, multiply the source feature vector by the corresponding projection weight matrix to obtain the projected feature vector of the target dimension. The initial values ​​of the elements of each projection weight matrix are randomly initialized using a normal distribution and optimized and updated through backpropagation during the model training phase.

[0082] S343: Stack the four projected feature vectors row by row to form a feature matrix to be fused. The feature matrix to be fused has four rows, corresponding to the four feature sources; the number of columns is the target dimension of the unified feature space.

[0083] S35: Perform multi-head attention mechanism calculation based on the feature matrix to be fused to generate a fused feature vector.

[0084] Specifically, the calculation process of the multi-head attention mechanism is as follows: S351: Set the number of attention heads. This number is determined based on the dimension of the feature matrix to be fused and computational resource constraints, and is usually set to four or eight heads. Divide the target dimension of the unified feature space by the number of attention heads to obtain the subspace dimension corresponding to each attention head.

[0085] S352: Set up a query projection matrix, key projection matrix, and value projection matrix for each attention head. Multiply the feature matrix to be fused with the query projection matrix, key projection matrix, and value projection matrix respectively to obtain the query matrix, key matrix, and value matrix corresponding to that attention head. The number of rows in each projection matrix is ​​the target dimension of the unified feature space, and the number of columns is the subspace dimension corresponding to each attention head.

[0086] S353: Calculate the attention weight matrix for each attention head. The calculation method for the attention weight matrix is ​​as follows: multiply the query matrix by the transpose of the key matrix to obtain the original attention score matrix; scale each element of the original attention score matrix by the square root of the subspace dimension; perform a normalized exponential function operation on the scaled matrix row by row to make the sum of the elements in each row equal to one, thus obtaining the attention weight matrix of that attention head.

[0087] S354: Calculate the weighted matrix for each attention head. The weighted matrix is ​​calculated by multiplying the attention weight matrix by the value matrix to obtain the weighted matrix for that attention head. The number of rows in the weighted matrix is ​​the same as the number of rows in the feature matrix to be fused, and the number of columns is the subspace dimension.

[0088] S355: Concatenate the weighted matrices of all attention heads column by column to restore the target dimension of the unified feature space, forming a multi-head attention output matrix.

[0089] S356: Perform residual concatenation and layer normalization on the multi-head attention output matrix. The residual concatenation method is to add the multi-head attention output matrix to the feature matrix to be fused element by element; the layer normalization method is to calculate the mean and standard deviation of the matrix after residual concatenation row by row, and subtract the mean of each row from each element and then divide by the standard deviation of that row.

[0090] S357: Perform global average pooling on the normalized matrix to compress the four rows of feature vectors into one row, obtaining a fused feature vector. The global average pooling method is as follows: calculate the arithmetic mean of each column of the matrix, and combine the column means into a one-dimensional vector. The dimension of the fused feature vector is the same as the target dimension of the unified feature space, and it contains comprehensive information of attention-weighted multi-band acoustic features and load features.

[0091] Specifically, the reason for dividing the pure impact acoustic spectrum feature matrix into three sub-intervals according to frequency bands in this step is that the acoustic responses of different types of expansion joint defects are mainly distributed in different frequency ranges. The decrease in elasticity caused by the aging of rubber strips is mainly manifested as changes in low-frequency vibration characteristics, the stiffness degradation caused by anchorage damage is mainly manifested as abnormal mid-frequency response, while bolt loosening and metal component impact mainly produce high-frequency transient responses. If all frequency band features are treated with equal weight, the feature signals of different types of defects may mask each other, reducing the discriminative ability of the classification model. Through frequency band division and dynamic weighting of the multi-head attention mechanism, the model can automatically adjust the degree of attention to each frequency band according to the current input sample. For samples with significant high-frequency anomalies, the weight of the high-frequency band is increased, and for samples with significant low-frequency anomalies, the weight of the low-frequency band is increased, thereby enhancing the expression intensity of defect-related features. The necessity of introducing load-encoded feature vectors and fusing them with acoustic features in a unified feature space lies in the fact that lightly loaded vehicles generate weaker impact excitation when passing through expansion joints. Even if defects exist, their acoustic response amplitude may be lower than that of heavily loaded vehicles passing normally. Judging solely based on the absolute value of acoustic features would lead to missed detections. By encoding axle load information and inputting it into the attention mechanism along with acoustic features, the model can learn the correlation between axle load and acoustic response, achieving implicit load normalization at the feature level. This allows classification decisions to be based on the acoustic response characteristics under unit excitation rather than the absolute response amplitude, thereby eliminating the interference of different excitation intensities on defect identification results.

[0092] S4: Input the fused feature vector into the pre-trained deep residual shrinking network, compare the topological distance with the disease voiceprint benchmark database, output the disease type and corresponding confidence level, and trigger the early warning strategy.

[0093] This step addresses the problem that existing monitoring technologies can only identify overall anomalies but cannot distinguish specific disease types. Based on fused feature vectors and a pre-built disease voiceprint benchmark library, it uses feature refinement of deep residual shrinkage networks and topological distance comparison mechanisms to output accurate disease type classification results and corresponding confidence levels, and triggers differentiated early warning strategies based on the confidence level.

[0094] Specifically, the process of outputting the disease type and corresponding confidence level and triggering the early warning strategy includes: S41: Construct a disease voiceprint benchmark library to store standard feature templates for various typical disease states.

[0095] Specifically, the construction process of the disease voiceprint reference library is as follows: S411: Determine the set of defect categories included in the defect soundprint benchmark library. The defect category set includes five categories: normal state, concrete breakage in the anchorage zone, beam fracture, rubber strip peeling, and bolt loosening. Normal state indicates that all components of the expansion joint are intact and functioning normally; concrete breakage in the anchorage zone indicates that the concrete in the connection area between the expansion joint and the bridge deck has cracked or fallen off; beam fracture indicates that the steel beam of the expansion joint has cracked or broken; rubber strip peeling indicates that the adhesion between the sealing rubber strip and the steel beam has failed or the rubber material has aged and cracked; bolt loosening indicates that the preload of the connecting bolts has decreased or the bolts have fallen off.

[0096] S412: Collect labeled sample data for each type of defect. The labeled sample data comes from clearly diagnosed defect cases in historical inspection records. Each case includes sound field tensor data collected on-site and manually labeled defect category tags. For the normal state category, samples collected under different traffic conditions are selected from expansion joints that have been confirmed to be defect-free during regular inspections. For each defect category, samples collected before maintenance are selected, and the defect type recorded in the maintenance work order is used as the tag.

[0097] S413: Perform the processing steps described in S2 and S3 sequentially on the labeled sample data of each category to generate the fusion feature vector corresponding to each sample.

[0098] S414: Perform cluster center calculation on all fused feature vectors within each disease category to obtain the standard feature template for that category. The method for calculating the cluster centers is as follows: calculate the element-wise arithmetic mean of all fused feature vectors for that category, and use the average vector as the standard feature template for that category.

[0099] S415: Store the standard feature templates for the five disease categories by category index to form a disease voiceprint benchmark library. The disease voiceprint benchmark library is a structured data table containing category labels and corresponding standard feature templates, supporting retrieval and comparison operations by category label.

[0100] S42: Construct a deep residual shrinking network and complete offline pre-training for the refinement of fused feature vectors.

[0101] Specifically, the structure and training process of the deep residual shrinkage network are as follows: S421: Design the hierarchical structure of the deep residual shrinking network. The deep residual shrinking network consists of an input layer, multiple stacked residual shrinking modules, and an output layer. The input layer receives the fused feature vector and performs batch normalization; each residual shrinking module contains two fully connected layers, a soft thresholding submodule, and an identity skip connection; the output layer maps the output of the last residual shrinking module to a refined feature vector with the same dimension as the fused feature vector. The number of residual shrinking modules is determined based on the balance between feature complexity and overfitting risk, typically set to three to five.

[0102] S422: Design the internal structure of the soft thresholding submodule. The input of the soft thresholding submodule is the output vector of the two fully connected layers in the residual shrinkage module. First, the mean of the input vector is calculated through a global average pooling layer. Then, the mean is mapped to a threshold scaling factor between zero and one through two small fully connected layers and an activation function. The threshold scaling factor is multiplied by the maximum absolute value of the input vector to obtain the adaptive threshold. Soft thresholding is performed on the input vector, using the same method as described in S234, and the output is a sparsified feature vector. The soft thresholding submodule can automatically suppress redundant feature components unrelated to the disease and enhance the relative strength of discriminative features.

[0103] S423: Offline pre-training of the deep residual shrinking network is performed using labeled sample data. The pre-training method is as follows: the fused feature vectors corresponding to the labeled samples of each category are used as input, and the disease category labels of the samples are used as supervision signals; a temporary classification head is added after the output layer to map the refined feature vectors to the predicted probability distributions of the five categories; a cross-entropy loss function is defined to measure the difference between the predicted probability distributions and the true labels; the stochastic gradient descent algorithm is used to minimize the loss function and update all learnable parameters in the network; training stops when the classification accuracy on the validation set no longer improves. After training, the temporary classification head is removed, and the network parameters from the input layer to the output layer are retained for online feature refinement.

[0104] S43: Input the fused feature vector into the pre-trained deep residual shrinking network to obtain the refined feature vector, and perform topological distance comparison with the disease voiceprint benchmark database.

[0105] Specifically, the topological distance comparison process is as follows: S431: Input the fusion feature vector generated in the current monitoring period into the deep residual shrinkage network. After layer-by-layer processing by each residual shrinkage module, the refined feature vector is obtained from the output layer.

[0106] S432: Read the standard feature templates for five disease categories sequentially from the disease voiceprint reference database, and calculate the cosine distance between the refined feature vector and each standard feature template. The cosine distance is calculated as follows: calculate the inner product of the refined feature vector and the standard feature template; calculate the magnitude of the refined feature vector and the magnitude of the standard feature template; divide the inner product by the product of the two magnitudes to obtain the cosine similarity; subtract the cosine similarity from one to obtain the cosine distance. The cosine distance ranges from zero to two; the smaller the distance, the closer the directions of the two vectors are.

[0107] S433: Search for the minimum value among the five cosine distance values, and use the disease category corresponding to the minimum distance as the predicted disease type for the current sample.

[0108] S434: Calculate the confidence score based on the minimum cosine distance value. The confidence score is calculated as follows: subtract the minimum cosine distance from one and divide by two to obtain the initial confidence score, which ranges from zero to one; calculate the difference between the minimum cosine distance and the second smallest cosine distance as the discrimination index; and use the weighted combination of the initial confidence score and the discrimination index as the final confidence score, with the weighting coefficients determined based on calibration analysis of historical prediction results. The closer the confidence score is to one, the higher the reliability of the prediction result.

[0109] S44: Implement a graded early warning strategy based on predicted disease type and confidence score.

[0110] Specifically, the execution process of the tiered early warning strategy is as follows: S441: Set confidence level thresholds and warning level mapping rules. Divide the confidence score interval into three levels: high confidence, medium confidence, and low confidence. The thresholds are determined based on the accuracy statistics of historical prediction results. For predicted disease types as normal, no warning is triggered regardless of the confidence level. For predicted disease types as abnormal, the warning level is determined based on the severity of the disease type and the confidence level.

[0111] S442: Establish a severity scoring table for different types of defects. This table maps five defect categories to severity scores: normal condition corresponds to zero points, rubber strip peeling corresponds to one point, bolt loosening corresponds to two points, concrete breakage in the anchorage zone corresponds to three points, and beam fracture corresponds to four points. The severity scores reflect the degree to which the defects affect the structural safety and functionality of the expansion joint.

[0112] S443: Calculate the comprehensive early warning index. The comprehensive early warning index is calculated by multiplying the severity score corresponding to the predicted disease type by the confidence score. The comprehensive early warning index considers both the severity of the disease type and the reliability of the prediction results.

[0113] S444: Trigger differentiated early warning strategies based on the comprehensive early warning index. When the comprehensive early warning index is less than the first early warning threshold, the system records monitoring data but does not issue an early warning, only accumulating observations in the background; when the comprehensive early warning index is greater than or equal to the first early warning threshold and less than the second early warning threshold, the system issues a yellow early warning, pushes early warning information to the maintenance platform, and suggests focusing on checking the expansion joint during the next planned inspection; when the comprehensive early warning index is greater than or equal to the second early warning threshold, the system issues a red early warning, simultaneously pushing emergency early warning information to the maintenance platform and on-site duty personnel, and suggests arranging a special inspection or implementing traffic control measures as soon as possible. The first and second early warning thresholds are set according to the maintenance department's risk tolerance and emergency response capabilities. See also Figure 7 This is a schematic diagram of a graded early warning trigger provided in an embodiment of this application. As shown in the figure, a horizontal scale bar is divided into three color intervals, corresponding to "no warning" (green), "yellow warning" (yellow), and "red warning" (red) from left to right, separated by two vertical threshold dashed lines. An indicator arrow points to a position on the scale bar, representing the calculated current "comprehensive early warning index" value. This index is obtained by multiplying the "severity score" of the predicted disease type by the "confidence score" output by the model. This application abandons the traditional "abnormal / normal" binary alarm mode and adopts a differentiated early warning strategy based on risk rating. Its beneficial effects are reflected in two aspects: First, scientificity: the early warning decision integrates information from two dimensions: "what is the disease" (type severity) and "how accurate is the judgment" (confidence). For example, even if the system judges it as a serious disease (such as beam fracture) with a low confidence level, its comprehensive index may not trigger a high-level early warning, which avoids overreaction caused by accidental misjudgment by the model. Second, practicality: graded early warning provides a clear and operable action guide for bridge maintenance departments. A "yellow alert" indicates that the system will be included in the next planned inspection and is considered preventative maintenance; a "red alert" requires immediate special inspections or temporary measures, and is considered emergency response. This tiered response mechanism ensures that serious risks are captured and addressed in a timely manner, optimizes maintenance resources, and avoids wasting manpower due to frequent false alarms, achieving a leap from "simple alarms" to "intelligent decision support."

[0114] S445: Generate an early warning report and store it in the historical database. The early warning report includes the following fields: monitoring timestamp, expansion joint number, vehicle parameter information that triggered the identification, predicted damage type, confidence score, comprehensive early warning index, early warning level, and recommended handling measures. The early warning report is stored in the historical database in structured record form for the maintenance department to query, analyze, and formulate maintenance plans.

[0115] For example, in a monitoring scenario of expansion joints on an elevated highway in a certain city, when a heavy truck passes through the monitoring section, the system collects multi-channel sound pressure signals and vehicle video data. After processing in S1, sound field tensor data is constructed, where the vehicle velocity vector shows that the vehicle passes through the expansion joint at a speed of 15 meters per second. In processing in S2, the system calculates the Doppler frequency shift compensation coefficient based on the velocity vector, which is approximately 1.045. After performing frequency axis inverse correction on each channel signal, adaptive wavelet denoising is performed to generate a clean impact sound spectrum feature matrix. In processing in S3, the system divides the clean impact sound spectrum feature matrix into three frequency bands and extracts aggregated features. The vehicle axle load information is encoded and input together with the acoustic features into a multi-head attention module. The attention weight distribution shows that high-frequency features receive higher attention weights. In S4 processing, the fused feature vector is refined by a deep residual shrinkage network and compared with the defect soundprint benchmark library. The calculated cosine distance with the standard template of bolt loosening category is the smallest, the defect type is predicted to be bolt loosening, the confidence score is 0.87, the comprehensive early warning index calculation result exceeds the first early warning threshold but is lower than the second early warning threshold, the system triggers a yellow early warning and pushes early warning information including the location of expansion joint, defect type and suggested maintenance priority to the maintenance platform.

[0116] Specifically, the necessity of using a deep residual shrinking network to refine the fused feature vector in this step lies in the fact that although the fused feature vector has integrated multi-band acoustic features and load features, it may still contain redundant information unrelated to disease type discrimination. This redundant information will reduce the accuracy of subsequent distance comparisons. The deep residual shrinking network automatically learns the importance of each feature component and suppresses redundant components through a soft thresholding submodule. At the same time, it ensures the training stability of the deep network through residual skip connections, enabling the refined feature vector to more prominently express disease-related discriminative features. Using a disease acoustic signature benchmark library for topological distance comparison instead of directly using the classifier output has the following advantages: the standard feature templates stored in the benchmark library are interpretable, allowing maintenance personnel to view the feature distribution of each category of templates to understand the classification basis. Furthermore, the benchmark library supports incremental updates; when a new disease type is discovered or more labeled samples are obtained, new standard feature templates can be directly added to the benchmark library without retraining the entire network. The tiered early warning strategy avoids the one-size-fits-all problem caused by simply relying on threshold judgments by comprehensively considering the severity of disease types and confidence scores. This makes the early warning results more in line with actual maintenance needs, provides tiered decision support for municipal maintenance departments, and realizes the leap from general abnormal alarms to precise classification diagnosis and differentiated early warning.

[0117] Example 2: This embodiment, based on Embodiment 1, provides a real-time monitoring system for the service status of bridge expansion joints based on voiceprint feature recognition, such as... Figure 8 As shown, it includes: Sound field tensor data construction module: used to collect raw sound pressure signals and video stream data when vehicles pass through gaps, extract multi-channel sound field data through a directional microphone array, combine video recognition to obtain vehicle parameters, and construct sound field tensor data; Pure acoustic spectrum feature generation module: used to perform Doppler frequency shift correction based on vehicle velocity vector in sound field tensor data, and to use an adaptive wavelet threshold algorithm to remove background noise and generate a pure impact acoustic spectrum feature matrix. Fusion Feature Vector Output Module: This module maps the pure impact sound spectrum feature matrix and vehicle axle load information to a unified feature space, uses a multi-head attention mechanism to assign dynamic weights to different frequency bands, and outputs a fusion feature vector. Disease identification and early warning module: It is used to input the fused feature vector into the pre-trained deep residual shrinking network, compare the topological distance with the disease voiceprint benchmark library, output the disease type and corresponding confidence level, and trigger the early warning strategy.

Claims

1. A method for real-time monitoring of the service status of bridge expansion joints based on voiceprint feature recognition, characterized in that, The method includes: S1: Collect raw sound pressure signals and video stream data when the vehicle passes through the gap, extract multi-channel sound field data through a directional microphone array, combine video recognition to obtain vehicle parameters, and construct sound field tensor data; S2: Doppler frequency shift correction is performed based on the vehicle velocity vector in the sound field tensor data, and an adaptive wavelet threshold algorithm is used to remove background noise and generate a clean impact sound spectrum feature matrix. S3: Map the pure impact sound spectrum feature matrix and vehicle axle load information to a unified feature space, use a multi-head attention mechanism to assign dynamic weights to different frequency bands, and output a fused feature vector. S4: Input the fused feature vector into the pre-trained deep residual shrinking network, compare the topological distance with the disease voiceprint benchmark database, output the disease type and corresponding confidence level, and trigger the early warning strategy.

2. The method for real-time monitoring of the service status of bridge expansion joints based on voiceprint feature recognition according to claim 1, characterized in that, The process of constructing sound field tensor data includes: Deploy a directional microphone array and collect raw sound pressure signals, simultaneously collect video stream data, and construct a raw multimodal acquisition data set; Based on the video stream data in the original multimodal acquisition dataset, vehicle parameter vectors are extracted using vehicle detection and tracking algorithms; Based on the eight-channel raw sound pressure sampling sequence in the original multimodal acquisition data set, the sound source azimuth sequence is calculated using a beamforming algorithm; Sound field tensor data is constructed based on the eight-channel original sound pressure sampling sequence, the sound source azimuth sequence, and the vehicle parameter vector.

3. The method for real-time monitoring of the service status of bridge expansion joints based on voiceprint feature recognition according to claim 2, characterized in that, The process for extracting the vehicle parameter vector is as follows: Perform object detection on each frame of the video stream data to identify the location of vehicle bounding boxes and vehicle categories in the images; Perform multi-target tracking on the vehicle bounding boxes detected in consecutive frames, establish the association between the same vehicle in different frames, and generate vehicle trajectory sequences; Calculate vehicle velocity vectors based on vehicle trajectory sequences; Vehicle axle load information is estimated based on vehicle category labels and vehicle bounding box dimensions.

4. The method for real-time monitoring of the service status of bridge expansion joints based on voiceprint feature recognition according to claim 1, characterized in that, The process for generating a pure impact acoustic spectrum feature matrix includes: The motion parameters required for Doppler correction are extracted from the sound field tensor data, and a Doppler frequency shift compensation coefficient sequence is constructed. Based on the Doppler frequency shift compensation coefficient sequence, inverse frequency axis correction is performed on the multi-channel sound pressure signal in the sound field tensor data to obtain the Doppler-corrected sound pressure tensor. Based on the Doppler-corrected sound pressure tensor, an adaptive wavelet threshold algorithm is used to perform background noise stripping to obtain the denoised pressure tensor. Based on the noise reduction tensor, Mel frequency cepstral coefficients are extracted to construct a pure impact acoustic spectrum feature matrix.

5. The method for real-time monitoring of the service status of bridge expansion joints based on voiceprint feature recognition according to claim 4, characterized in that, The processing flow of the adaptive wavelet threshold algorithm is as follows: The Doppler-corrected sound pressure tensor is processed sequentially by channel and time window. For each one-dimensional sound pressure sequence to be processed, a multi-scale discrete wavelet transform is performed to decompose the signal into approximation coefficients and detail coefficients at multiple scales. Calculate the noise standard deviation estimate for the detail coefficients at each scale; Adaptive thresholds for each scale are calculated based on noise standard deviation estimates and signal length. Soft thresholding is applied to detail coefficients at each scale; Based on the processed approximation coefficients and detail coefficients at each scale, an inverse discrete wavelet transform is performed to reconstruct the multi-scale decomposition results into a time-domain signal, resulting in a denoised sound pressure sequence.

6. The method for real-time monitoring of the service status of bridge expansion joints based on voiceprint feature recognition according to claim 1, characterized in that, The process of outputting the fused feature vector includes: Perform frequency band partitioning on the pure impact acoustic spectrum feature matrix to construct a set of multi-frequency band feature sub-matrices; Perform time-dimensional aggregation on each submatrix in the multi-band feature submatrix set to obtain a multi-band feature vector set; Axle load information is extracted from the vehicle parameter vector attached to the pure impact sound spectrum feature matrix to construct a load coding feature vector; The multi-band feature vector set and the payload coding feature vector are mapped to a unified feature space to construct the feature matrix to be fused. Multi-head attention mechanism is used to calculate and generate fused feature vectors based on the feature matrix to be fused.

7. The method for real-time monitoring of the service status of bridge expansion joints based on voiceprint feature recognition according to claim 6, characterized in that, The calculation process of the multi-head attention mechanism is as follows: Set the number of attention heads, divide the target dimension of the unified feature space by the number of attention heads, and obtain the subspace dimension corresponding to each attention head; For each attention head, a query projection matrix, a key projection matrix, and a value projection matrix are set. The feature matrix to be fused is multiplied by the query projection matrix, the key projection matrix, and the value projection matrix respectively to obtain the query matrix, key matrix, and value matrix corresponding to that attention head. Calculate the attention weight matrix for each attention head; Calculate the weighting matrix for each attention head; By concatenating the weighted matrices of all attention heads column by column, the target dimension of the unified feature space is restored, forming a multi-head attention output matrix; The multi-head attention output matrix is ​​subjected to residual connection and layer normalization. Global average pooling is then performed on the normalized matrix to compress the four rows of feature vectors into one row, resulting in a fused feature vector.

8. The method for real-time monitoring of the service status of bridge expansion joints based on voiceprint feature recognition according to claim 1, characterized in that, The process of outputting the disease type and corresponding confidence level and triggering the early warning strategy includes: Construct a disease voiceprint benchmark library to store standard feature templates for various typical disease states; A deep residual shrinking network was constructed and pre-trained offline for the refinement of fused feature vectors; The fused feature vectors are input into a pre-trained deep residual shrinking network to obtain refined feature vectors, which are then compared topologically with a disease soundprint benchmark database. A tiered early warning strategy is implemented based on the predicted disease type and confidence score.

9. The method for real-time monitoring of the service status of bridge expansion joints based on voiceprint feature recognition according to claim 8, characterized in that, The structure and training process of the deep residual shrinkage network are as follows: The deep residual shrinkage network is designed with a hierarchical structure consisting of an input layer, multiple stacked residual shrinkage modules, and an output layer. The input layer receives the fused feature vector and performs batch normalization. Each residual shrinkage module contains two fully connected layers, a soft thresholding submodule, and an identity jump connection. The output layer maps the output of the last residual shrinkage module to a refined feature vector with the same dimension as the fused feature vector. The internal structure of the soft thresholding submodule is designed. The input of the soft thresholding submodule is the output vector of two fully connected layers in the residual shrinkage module. First, the mean of the input vector is calculated through a global average pooling layer. Then, the mean is mapped to a threshold scaling factor between zero and one through two small fully connected layers and an activation function. The threshold scaling factor is multiplied by the maximum value of the absolute value of the input vector to obtain the adaptive threshold. Soft thresholding is performed on the input vector, and the output is a sparsed feature vector. Offline pre-training of a deep residual shrinking network is performed using labeled sample data. The fused feature vectors corresponding to the labeled samples of each category are used as input, and the disease category labels of the samples are used as supervision signals. A temporary classification head is added after the output layer to map the refined feature vectors to the predicted probability distributions of the five categories. A cross-entropy loss function is defined to measure the difference between the predicted probability distributions and the true labels. The stochastic gradient descent algorithm is used to minimize the loss function and update all learnable parameters in the network. Training stops when the classification accuracy on the validation set no longer improves. After training, the temporary classification head is removed, and the network parameters from the input layer to the output layer are retained for online feature refinement.

10. A real-time monitoring system for the service status of bridge expansion joints based on voiceprint feature recognition, used to implement the real-time monitoring method for the service status of bridge expansion joints based on voiceprint feature recognition as described in any one of claims 1-9, characterized in that, The system includes: Sound field tensor data construction module: used to collect raw sound pressure signals and video stream data when vehicles pass through gaps, extract multi-channel sound field data through a directional microphone array, combine video recognition to obtain vehicle parameters, and construct sound field tensor data; Pure acoustic spectrum feature generation module: used to perform Doppler frequency shift correction based on vehicle velocity vector in sound field tensor data, and to use an adaptive wavelet threshold algorithm to remove background noise and generate a pure impact acoustic spectrum feature matrix. Fusion Feature Vector Output Module: This module maps the pure impact sound spectrum feature matrix and vehicle axle load information to a unified feature space, uses a multi-head attention mechanism to assign dynamic weights to different frequency bands, and outputs a fusion feature vector. Disease identification and early warning module: It is used to input the fused feature vector into the pre-trained deep residual shrinking network, compare the topological distance with the disease voiceprint benchmark library, output the disease type and corresponding confidence level, and trigger the early warning strategy.

Images