Municipal road state sensing method and system based on sensor fusion

Abstract

The invention discloses a municipal road state sensing method and system based on sensor fusion, and the method comprises the steps: collecting multi-modal road information data from a sensing network along a municipal road, and generating a standardized road feature vector; the method comprises the following steps: extracting frequency domain characteristics of pavement bearing capacity through time-frequency analysis, establishing a coupling model of vehicle load and pavement response, and calculating various road state indexes; outputting a structure health index and marking a thermodynamic diagram of a key damage position; real-time traffic flow data and meteorological environment parameters are fused, and an early warning level adaptive adjustment model based on fuzzy reasoning is established; and when a risk entropy value output by the early warning level adaptive adjustment model exceeds a dynamic threshold value, triggering a multi-level collaborative maintenance decision scheme and generating a visual road state digital twinborn body. According to the invention, the problem of insufficient road state sensing precision in a complex environment is solved, and dynamic evaluation and intelligent early warning of the full life cycle health state are realized.

Description

Technical Field

[0001] The present invention relates to the fields of intelligent transportation, Internet of Things and artificial intelligence technology, and in particular to a municipal road state perception method and system based on sensor fusion. Background Art

[0002] With the acceleration of urbanization, the health of municipal roads is directly related to the safety and stability of the city. Traditional road monitoring technology has many limitations and is difficult to meet the needs of modern urban road management. On the one hand, existing monitoring methods mostly rely on a single sensor and cannot fully capture the multi-dimensional status information of the road, resulting in one-sided monitoring data and difficulty in accurately reflecting the real situation of the road. On the other hand, traditional monitoring systems have deficiencies in data processing and analysis, lack of effective data fusion and deep mining capabilities, and difficulty in extracting valuable key information from massive monitoring data, and thus unable to achieve accurate evaluation and prediction of road conditions.

[0003] In addition, existing technologies often show poor adaptability when facing complex and changing environmental conditions, which will seriously affect the accuracy and reliability of monitoring data. Moreover, traditional monitoring methods are also lacking in risk warning and decision support. They cannot issue warnings in a timely and accurate manner based on real-time monitoring data and provide effective maintenance decision-making plans. It is difficult to effectively prevent the occurrence and development of road diseases and ensure the safe operation of roads. Therefore, there is an urgent need for a municipal road status perception method and system based on sensor fusion to overcome the shortcomings of existing technologies and achieve comprehensive, accurate, real-time perception and effective management of municipal road status. Summary of the invention

[0004] The present invention provides a municipal road state perception method based on sensor fusion, comprising: S1. Collect multimodal road information data from the perception network along the municipal roads, and process the road information data to generate standardized road feature vectors; S2. Based on the standardized road characteristic vector, the frequency domain characteristics of the road surface bearing capacity are extracted through time-frequency analysis, a coupling model of vehicle load and road surface response is established, and various road status indicators are calculated; S3, determine the influence weight of the damage factor of each road condition indicator, output the structural health index and mark the heat map of the key damage location; S4. Based on the heat map of key damage locations, the diffusion path of damage factors is modeled using a spatiotemporal graph convolutional network, and real-time traffic flow data and meteorological environmental parameters are integrated to establish an adaptive adjustment model for warning levels based on fuzzy reasoning. S5. When the risk entropy value output by the warning level adaptive adjustment model exceeds the dynamic threshold, a multi-level collaborative maintenance decision-making plan is triggered and a visualized road status digital twin is generated.

[0005] The municipal road state perception method based on sensor fusion as described above, wherein multimodal road information data is collected from a perception network along the municipal road, comprises the following sub-steps: The three-dimensional angular velocity and linear acceleration data are collected through the nine-axis inertial measurement unit, and the quaternion attitude solution algorithm is used to compensate for the point cloud coordinate offset caused by the vibration of the laser radar equipment, establish the rotation matrix from the carrier coordinate system to the world coordinate system, and calculate the Euler angle deviation between adjacent scan lines; When building a time synchronization model at the 5G edge node, a two-way timestamp exchange protocol is used to calibrate the clock deviation of each sensor. The spatial point cloud data of the lidar and the velocity vector information of the millimeter-wave radar are fused through an extended Kalman filter to generate a multimodal data set aligned in time and space.

[0006] A municipal road state perception method based on sensor fusion as described above, wherein various road state indicators are calculated, including: constructing a dual-channel neural network to process three-dimensional point clouds and infrared images based on standardized road feature vectors, extracting three-dimensional geometric features of cracks through the point cloud channel, and calculating crack width, depth and direction parameters, including the following sub-steps: The point cloud channel uses a three-dimensional sparse convolutional network to extract crack features, voxelizes the original point cloud, calculates the variance of the normal vector distribution of each voxel, and captures the irregular geometric features of the crack edge through a deformable convolution kernel. When processing the infrared channel, an improved deep semantic segmentation network is used to segment the temperature anomaly area, and the heat conduction equation is combined to invert and calculate the internal defect depth of the material. A surface temperature field model considering the solar radiation absorption rate and the emissivity of the pavement material is established, and the implicit difference format of the three-dimensional heat conduction equation is solved.

[0007] As described above, a municipal road state perception method based on sensor fusion is described, wherein, when processing the infrared channel, an improved deep semantic segmentation network is used to segment the temperature anomaly area, and the internal defect depth of the material is calculated by combining the heat conduction equation inversion, and a surface temperature field model considering the solar radiation absorption rate and the emissivity of the pavement material is established. The implicit difference format of the three-dimensional heat conduction equation is solved, including the following sub-steps: Collect the time series temperature data of the infrared thermal imager and construct a boundary condition matrix including ambient temperature, wind speed and sunshine intensity; The conjugate gradient method is used to solve the inverse problem of unsteady-state heat conduction and iteratively optimize the distribution of equivalent thermal conductivity of internal defects in the material. The influence of temperature measurement noise on the inversion results is eliminated through regularization processing, and the isothermal gradient cloud map of the defect area is generated.

[0008] A municipal road state perception method based on sensor fusion as described above, wherein based on the structural health index and the heat map with key damage locations marked, a spatiotemporal graph convolutional network is used to model the diffusion path of the damage factor, and real-time traffic flow data and meteorological environment parameters are integrated to calculate the risk field intensity attenuation function, including the following sub-steps: Define the spatiotemporal propagation model of damage diffusion and establish the state transfer equation including crack length, traffic load frequency and temperature cycle number; The random walk algorithm is used to simulate the propagation path of damage factors in the road network and calculate the superposition intensity of the risk field of each road section; The fatigue characteristics of pavement materials are introduced to correct the attenuation coefficient, and an exponential attenuation model considering material aging is constructed.

[0009] The above-mentioned municipal road state perception method based on sensor fusion, wherein a spatiotemporal propagation model of damage diffusion is defined, and a state transfer equation including crack length, traffic load frequency and temperature cycle number is established, includes the following sub-steps: Extract the correlation characteristics between crack growth rate and traffic flow in historical damage data and construct a hidden Markov state transition probability matrix; The precipitation and freeze-thaw cycle data from meteorological stations are integrated to calculate the acceleration factor of environmental factors on the state transition rate.

[0010] A municipal road state perception method based on sensor fusion as described above, wherein an adaptive adjustment model of warning level based on fuzzy reasoning is established, and when the risk entropy value exceeds the dynamic threshold, a multi-level collaborative maintenance decision-making scheme is triggered and a visualized road state digital twin is generated, including the following sub-steps: Four warning levels are divided according to the structural health index, and a dynamic threshold trigger mechanism is set: when the index is lower than the threshold for consecutive preset hours, the second-level response is initiated, and when two or more critical damages occur at the same time, the first-level response is triggered; A multi-agent simulation module is embedded in the digital twin to simulate the impact of different maintenance plans on traffic flow, establish a vehicle lane-changing model in the maintenance construction area, and calculate the capacity attenuation rate of the road section during the construction period.

[0011] The present invention also provides a municipal road state perception system based on sensor fusion, comprising: Data collection and preprocessing module: collect multimodal road information data from the perception network along municipal roads, process the road information data to generate standardized road feature vectors; Feature processing and damage assessment module: Based on the standardized road feature vector, the frequency domain characteristics of the road bearing capacity are extracted through time-frequency analysis, a coupling model of vehicle load and road response is established, and various road status indicators are calculated; the influence weight of the damage factor of each road status indicator is determined, and the structural health index is output and a heat map with key damage locations is marked; Decision support and visualization module: Based on the heat map of key damage locations, the diffusion path of damage factors is modeled using a spatiotemporal graph convolutional network, and real-time traffic flow data and meteorological environmental parameters are integrated to establish an adaptive adjustment model for warning levels based on fuzzy reasoning. When the risk entropy value output by the adaptive adjustment model of the warning level exceeds the dynamic threshold, a multi-level collaborative maintenance decision-making plan is triggered and a visualized road status digital twin is generated.

[0012] A computer storage medium, characterized in that it comprises: at least one memory and at least one processor; A memory for storing one or more program instructions; A processor is used to run one or more program instructions to execute any of the municipal road status perception methods based on sensor fusion described above.

[0013] The beneficial effects achieved by the present invention are as follows: The present invention solves the problem of insufficient accuracy in road state perception in complex environments, and realizes dynamic evaluation and intelligent early warning of the health status throughout the life cycle. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0015] Figure 1 This is a flow chart of a municipal road status perception method based on sensor fusion provided in an embodiment of the present application. DETAILED DESCRIPTION

[0016] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0017] Embodiment 1 like Figure 1 As shown, a municipal road state perception method based on sensor fusion in an embodiment of the present application includes: Step S1, collecting multimodal road information data from the perception network along the municipal roads, processing the road information data to generate a standardized road feature vector; Specifically, integrated sensing terminals integrating laser radar, millimeter-wave radar, infrared thermal imager and piezoelectric sensor are deployed along municipal roads. The built-in nine-axis inertial measurement unit collects equipment vibration data in real time, and compensates for the spatial distortion of laser point cloud through attitude solution algorithm. Microsecond time synchronization is achieved based on 5G edge nodes, and the extended Kalman filter is used to build a multi-sensor spatial coordinate conversion model to unify data from different perspectives into the global coordinate system. In response to complex environmental interference, the adversarial generative network is used to simulate abnormal modes such as sudden changes in illumination and electromagnetic noise. The reliability of each sensor data is dynamically evaluated by calculating the spatiotemporal consistency index to generate a standardized road feature vector, which specifically includes the following sub-steps: Step S11, collecting three-dimensional angular velocity and linear acceleration data through the nine-axis inertial measurement unit, using the quaternion attitude solution algorithm to compensate for the point cloud coordinate offset caused by the vibration of the laser radar equipment, establishing a rotation matrix from the carrier coordinate system to the world coordinate system, and calculating the Euler angle deviation between adjacent scan lines; Based on the high-precision data acquisition of the nine-axis inertial measurement unit, the system introduces a piezoelectric vibration sensor and a temperature compensation module to build a multi-source coupling error model. The original angular velocity and acceleration data of the nine-axis inertial measurement unit sampled at 1kHz are used to numerically integrate the quaternion differential equation using the following formula to update the attitude quaternion in real time:

[0018] in, Indicates the next moment The updated attitude quaternion is calculated by integration to obtain the new attitude result; Indicates the current time The attitude quaternion is used to characterize the attitude state of the current system; Indicates the time interval for data sampling; Represents the weighting coefficient of the fourth-order Runge-Kutta method, which improves the accuracy of numerical integration and reduces the cumulative error of attitude update; , , , They represent the intermediate variables of the iterative calculation of the Runge-Kutta method and the calculated values ​​of the slope of the quaternion differential equation at different time points.

[0019] At the same time, in view of the environmental temperature drift effect, a polynomial regression model of gyroscope zero bias and temperature is established to dynamically correct the sensor deviation. In order to further suppress high-frequency vibration noise, the lidar point cloud is mapped from the carrier coordinate system to the world coordinate system based on the Lie group framework, and the manifold optimization algorithm is used to solve the minimum point cloud matching residual. The noise distribution is adaptively adjusted through hybrid particle filtering, and the pre-integration constraints of the nine-axis inertial measurement unit and the geometric characteristics of the lidar are integrated to achieve robust attitude solution and point cloud spatiotemporal consistency calibration under vibration interference.

[0020] Step S12: When constructing a time synchronization model at the 5G edge node, a two-way timestamp exchange protocol is used to calibrate the clock deviation of each sensor, and the spatial point cloud data of the lidar and the velocity vector information of the millimeter-wave radar are fused through an extended Kalman filter to generate a multimodal data set aligned in time and space; When building a high-precision time synchronization model at the 5G edge node, the system dynamically calibrates the clock deviation of each sensor through a two-way timestamp exchange protocol. The edge node and the sensor periodically send synchronization signals carrying a quantum random number entropy source to each other, and use the symmetry of the two-way transmission delay to calculate the clock offset and propagation delay. The physical layer hardware clock is combined to calibrate the crystal oscillator frequency drift to ensure that the clocks of multiple devices are synchronized to microsecond accuracy.

[0021] In order to achieve deep fusion of lidar point cloud and millimeter-wave radar velocity vector, the extended Kalman filter framework is adopted. The three-dimensional position and velocity of the target are used as state quantities. The trajectory is predicted by the motion model and the spatial coordinate observation of the lidar and the Doppler velocity information of the millimeter-wave radar are fused. The observation matrix is ​​used to map the multimodal data to a unified state space for iterative optimization, and the spatiotemporal alignment error is dynamically corrected.

[0022] In response to complex environmental interference, a generative adversarial network is introduced to simulate abnormal modes such as sudden changes in illumination and electromagnetic noise, generate training samples containing disturbance signals, and analyze the consistency of sensor data in time series and spatial topological structure through the spatiotemporal consistency evaluation module. Reliability weights are dynamically assigned, abnormal data is removed, and missing information is compensated. Finally, a standardized and spatiotemporally aligned multimodal road feature vector is output:

[0023] in, represents the normalized road feature vector; represents a nonlinear projection function, which compresses the fused high-dimensional tensor into a low-dimensional feature vector; Represents the total number of time steps in the window; Represents the spatiotemporal attention module, calculating the moment and Features Attention between time and space, capturing the temporal and spatial correlation features; Indicates time feature The gradient reflects the magnitude of the feature change at that moment; Represents the smoothing term control parameter, which adjusts the degree of gradient smoothing. The larger the value, the weaker the suppression of gradient changes. Represents the gradient smoothing term, which suppresses the drastic changes in feature gradients through exponential operations, making feature changes smoother.

[0024] Step S2: extract the frequency domain characteristics of the road surface bearing capacity through time-frequency analysis based on the standardized road characteristic vector, establish a coupling model of vehicle load and road surface response, and calculate various road status indicators; Step S3, determine the influence weight of the damage factor of each road condition indicator, output the structural health index and mark the heat map of the key damage location; Specifically, based on the standardized road feature vector, a dual-channel neural network is constructed to process three-dimensional point clouds and infrared images. The point cloud channel extracts the three-dimensional geometric features of cracks and calculates the crack width, depth and direction parameters; the infrared channel segments the temperature anomaly area and inversely calculates the thermal conduction gradient distribution of internal defects in the material. The piezoelectric sensor data is integrated, and the frequency domain characteristics of the road bearing capacity are extracted through time-frequency analysis, and a coupling model of vehicle load and road response is established.

[0025] The three core indicators of crack propagation rate, material fatigue cumulative damage and roadbed settlement velocity are calculated comprehensively, and the influence weight of each damage factor is determined by using the grey correlation analysis method. Finally, the structural health index and the thermal map with key damage locations marked are output. The specific steps include the following: Step S21, the point cloud channel uses a three-dimensional sparse convolutional network to extract crack features, voxelizes the original point cloud, calculates the normal vector distribution variance of each voxel, and captures the irregular geometric features of the crack edge through a deformable convolution kernel; First, the original crack point cloud is preprocessed by multi-scale voxelization, and the space is divided into three-dimensional grids with a resolution of 0.5 cm. After removing outliers, the spatial distribution characteristics of the points in each voxel are counted. For the structured data after voxelization, a three-dimensional convolutional network based on sparse tensors is constructed, and the geometric topological features of the cracks are extracted through hierarchical convolution kernels: the first layer of convolution kernels captures local curvature changes, and the second layer introduces a deformable convolution module, and uses learnable offset parameters to dynamically adjust the shape of the convolution kernel receptive field to adapt to the irregular jagged shape of the crack edge; at the same time, the standard deviation of the point cloud normal vector direction is calculated in each voxel:

[0026] in, Represents the standard deviation of the point cloud normal vector direction; Represents the number of point clouds within a voxel; The index variable representing the sum operation represents each individual in the point cloud set; represents the normal vector of the i-th point cloud; Represents the average value of the point cloud normal vectors.

[0027] A normal vector distribution variance map is constructed and used as an auxiliary feature to input the network to enhance the sensitivity to the roughness of the crack surface. A multi-task loss function is used during training:

[0028] in, represents the total loss function, which is used to jointly optimize multiple tasks such as crack segmentation and geometric parameter regression. It consists of crack segmentation loss, geometric regression loss, asymmetric focal loss and regularization term; They are the hyperparameters related to crack segmentation loss, geometric regression loss, and asymmetric focal loss, which are used to adjust the weight of each part of the loss in the total loss; represents the set of all voxels; Voxel The true label value of ; Representing voxels The predicted probability value of Represents the positive sample voxel set The number of elements of ; represents the true geometric parameter value of voxel v, ; Voxel The predicted geometric parameter values ​​of Indicates smoothness Loss function, used to handle outliers in regression problems; Represents the balance factor, which is used to adjust the contribution of positive and negative samples to the loss; Represents the focus parameter, which controls the degree of attention paid to difficult-to-classify samples. The larger the value, the more attention the model pays to difficult-to-classify samples.

[0029] The crack segmentation accuracy and geometric parameter regression error are jointly optimized, and the negative sample imbalance problem is suppressed by asymmetric focal loss.

[0030] Step S22: When processing the infrared channel, an improved deep semantic segmentation network is used to segment the temperature anomaly area, and the depth of internal defects of the material is calculated by inversion of the heat conduction equation. A surface temperature field model considering the solar radiation absorption rate and the emissivity of the pavement material is established to solve the implicit difference format of the three-dimensional heat conduction equation. Step S221, collecting time series temperature data of the infrared thermal imager, and constructing a boundary condition matrix including ambient temperature, wind speed and sunshine intensity; The time series temperature field data of the road surface is collected by infrared thermal imager at a frequency of 10Hz, and the ambient temperature, wind speed sensor and sunshine radiometer of the meteorological station are connected synchronously to construct a boundary condition matrix aligned in time and space. The original infrared data is corrected for non-uniformity and radiant calibration to eliminate the thermal noise of the sensor itself; the discrete meteorological parameters are converted into continuous functions based on the time series interpolation method as the external boundary input of the heat conduction equation.

[0031] Step S222: using the conjugate gradient method to solve the inverse problem of non-steady-state heat conduction, and iteratively optimizing the distribution of equivalent thermal conductivity of internal defects of the material; Based on the temperature anomaly area segmented by the improved deep semantic segmentation network, a three-dimensional non-steady-state partial differential equation for heat conduction is established, with the surface temperature field as the boundary condition and the internal defects of the material equivalent to the spatial distribution of thermal conductivity. The implicit finite difference method is used to discretize the non-steady-state partial differential equation for heat conduction in time and space to generate a large sparse stiffness matrix; the preconditioned conjugate gradient method is used to iteratively solve the linear equations, the gradient of the objective function pair is calculated by the adjoint state method, and the thermal conductivity distribution is updated in combination with the optimizer until the mean square error between the inverted temperature field and the measured data converges.

[0032] Step S223: Eliminate the influence of temperature measurement noise on the inversion result through regularization processing, and generate an isothermal gradient cloud map of the defect area.

[0033] Aiming at the ill-posed problem caused by temperature measurement noise in the inversion process, the Tikhonov regularization term is introduced to constrain the smoothness of the solution and construct the objective function with regularization:

[0034] in, represents the regularized objective function; Indicates that based on parameters The model output results; Indicates the actual measured results; Represents a hyperparameter, which is used to balance the weights of the model fitting term and the regularization term, and control the parameters The strength of the constraint; Usually a matrix used to calculate the parameters Perform some kind of linear transformation; Indicates the parameters to be optimized; Representation parameters The prior value of is a known reference value, which is used to constrain the parameters and avoid overfitting.

[0035] By determining the optimal regularization parameter, the data fitting term and the prior constraints are balanced; the fast iterative shrinkage threshold algorithm is used to solve the optimization problem and suppress the influence of high-frequency noise on the defect depth estimation. Finally, the optimized thermal conductivity distribution is extracted by isosurface, and the isothermal gradient cloud map of the defect area is generated, marking the abnormal heat flow position and the internal defect geometric parameters.

[0036] Step S4: Based on the heat map of the key damage location, a spatiotemporal graph convolutional network is used to model the diffusion path of the damage factor, and the real-time traffic flow data and meteorological environment parameters are integrated to establish an adaptive adjustment model of the warning level based on fuzzy reasoning; Step S5: When the risk entropy value output by the warning level adaptive adjustment model exceeds the dynamic threshold, a multi-level collaborative maintenance decision-making plan is triggered and a visualized road status digital twin is generated.

[0037] Specifically, based on the structural health index and the heat map with key damage locations marked, the diffusion path of the damage factor is modeled using a spatiotemporal graph convolutional network, and the real-time traffic flow data and meteorological environment parameters are integrated to calculate the risk field intensity attenuation function; the stress distribution cloud map under different load conditions is simulated, and a road failure probability prediction model is constructed in combination with an improved extreme learning mechanism; a long short-term memory network with a double-layer attention mechanism is designed to dynamically integrate historical maintenance records and real-time monitoring data to generate a road remaining life curve; an adaptive adjustment model for the warning level based on fuzzy reasoning is established. When the risk entropy value exceeds the dynamic threshold, a multi-level collaborative maintenance decision-making plan is triggered and a visualized road status digital twin is generated, which specifically includes the following sub-steps: Step S31, defining a spatiotemporal propagation model of damage diffusion, and establishing a state transfer equation including crack length, traffic load frequency, and temperature cycle number; Step S311, extracting the crack growth rate and traffic flow correlation characteristics in the historical damage data, and constructing a hidden Markov state transition probability matrix; The crack length growth rate, daily average equivalent axle load times and monthly average temperature cycle times are extracted from the historical detection data, and the damage state is divided into three types of hidden states: (slight), (medium), (Serious). Construct a transition count matrix based on the state sequence, and calculate the transition probability matrix by dealing with the zero count problem:

[0038] in, Represents a 3×3 state transition probability matrix, which is used to describe the probability of a system transitioning from one state to another; Indicates that the probability value is non-negative, because the probability value range is [0,1]; For each row , all from the state The sum of the probabilities of transitioning to other states is 1, which means that starting from a certain state, it will inevitably transition to other states, and the total probability is 100%.

[0039] Step S312, integrating the precipitation and freeze-thaw cycle data of the meteorological station, and calculating the acceleration factor of the environmental factors on the state transfer rate; Integrate the precipitation and freeze-thaw cycle times of meteorological stations to define the environmental stress function ,in, The parameters of the model are constants; Indicates precipitation; Indicates the number of freeze-thaw cycles. Based on the accelerated life test data fitting model, calculate the environmental acceleration factor ,in, represents the environmental acceleration factor; It refers to the minimum energy required for reactant molecules to reach activated molecules in a chemical reaction. represents the Boltzmann constant, which is approximately , is an important constant linking microscopic energy and macroscopic temperature; Indicates the reference temperature, which is the temperature value used as a benchmark; is the ambient temperature; Indicates at time A certain environmental stress-related quantity at the time; yes The corresponding benchmark value; It is an exponential parameter related to environmental stress and is used to adjust the influence of environmental stress on the acceleration factor. The environmental acceleration factor is multiplied into the damage rate term of the state transfer equation to correct the influence of the environment on material aging.

[0040] Step S32: Use a random walk algorithm to simulate the propagation path of the damage factor in the road network and calculate the risk field superposition intensity of each road section.

[0041] Based on the modeling of the road network topology structure, the road sections are abstracted as graph nodes and the adjacency matrix is ​​defined. The propagation behavior of the damage factor is simulated by the random walk algorithm: the damage factor is initialized as a virtual particle, starting from the damaged road section node, the transfer probability is dynamically adjusted according to the adjacency relationship and the current damage level of the road section, the particle randomly walks in the road network and records the frequency of visits to each road section, and the risk field intensity of the road section is calculated in combination with the time attenuation factor: ,in, represents the intensity of the risk field; Indicates the upper limit of the sum, from 1 to Perform accumulation operation; Indicates The propagation paths pass through the road segment The number of times; Indicates road segment The current level of injury; It represents the attenuation parameter, which is used to measure the degree of risk attenuation with a certain factor; Indicates that the damage is along Paths propagated to road segments Finally, the influence weights of all particle paths are superimposed to generate a global risk heat map.

[0042] Step S33: introducing the pavement material fatigue characteristic correction attenuation coefficient to construct an exponential attenuation model that takes material aging into consideration.

[0043] The Miller cumulative damage theory is used to quantify the progressive effect of load cycles on material properties. A nonlinear aging factor related to service time is introduced, and the attenuation rate parameter is corrected to reflect the irreversible expansion of microcracks inside the material. The attenuation coefficient is dynamically updated in combination with real-time monitored temperature and humidity data, and a prediction model for the evolution of damage values ​​over time and environmental conditions is constructed.

[0044] Step S34: divide the warning levels into four categories according to the structural health index, and set a dynamic threshold trigger mechanism: when the index is lower than the threshold for three consecutive hours, the second-level response is initiated, and when two or more critical damages occur at the same time, the first-level response is triggered; Four warning levels are dynamically divided according to the structural health index, and a multi-condition trigger mechanism is designed: the sliding window is used to statistically calculate the mean and standard deviation of the historical structural health index, and the dynamic threshold is calculated to eliminate the impact of seasonal or sudden traffic changes. When the structural health index is continuously lower than the threshold, a time accumulation response is triggered, and a composite judgment is made based on the spatial distribution of key damage points. If both spatial aggregation and temporal continuity are met, the warning level is upgraded, and emergency response instructions are pushed in real time through the digital twin platform.

[0045] Step S35: embed a multi-agent simulation module in the digital twin to simulate the impact of different maintenance plans on traffic flow, including establishing a vehicle lane-changing model in the maintenance construction area and calculating the capacity attenuation rate of the road section during the construction period; A vehicle intelligent agent behavior model is established to describe the interactive logic of lane changing and following, and the geometric constraints and speed limit rules of the construction area are embedded. The traffic flow evolution scenario is generated through the Monte Carlo method, and the capacity decay rate and delay time under different closed lane schemes are quantified. Reinforcement learning is combined to optimize the construction window period and traffic diversion strategy, and a comprehensive benefit evaluation report of the maintenance plan is output.

[0046] Embodiment 2 Embodiment 2 of the present application provides a municipal road state perception system based on sensor fusion, including: Data collection and preprocessing module 21: collect multimodal road information data from the perception network along the municipal roads, process the road information data to generate a standardized road feature vector; Feature processing and damage assessment module 22: Based on the standardized road feature vector, the frequency domain characteristics of the road bearing capacity are extracted through time-frequency analysis, a coupling model of vehicle load and road response is established, and various road status indicators are calculated; the influence weight of the damage factor of each road status indicator is determined, and the structural health index is output and a heat map with key damage locations is marked; Decision support and visualization module 23: Based on the heat map of key damage locations, the diffusion path of damage factors is modeled using a spatiotemporal graph convolutional network, and real-time traffic flow data and meteorological environmental parameters are integrated to establish an adaptive adjustment model for warning levels based on fuzzy reasoning; when the risk entropy value output by the adaptive adjustment model for warning levels exceeds the dynamic threshold, a multi-level collaborative maintenance decision-making plan is triggered and a visualized road status digital twin is generated.

[0047] Corresponding to the above embodiment, an embodiment of the present invention provides a computer storage medium, including: at least one memory and at least one processor; The memory is used to store one or more program instructions; A processor is used to run one or more program instructions to execute a municipal road state perception method based on sensor fusion.

[0048] Corresponding to the above-mentioned embodiment, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer storage medium contains one or more program instructions, and the one or more program instructions are used by a processor to execute a municipal road state perception method based on sensor fusion.

[0049] The embodiments disclosed in the present invention provide a computer-readable storage medium, in which computer program instructions are stored. When the computer program instructions are executed on a computer, the computer executes the above-mentioned municipal road state perception method based on sensor fusion.

[0050] In the embodiment of the present invention, the processor may be an integrated circuit chip having the ability to process signals. The processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0051] The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiments of the present invention can be directly embodied as a hardware decoding processor for execution, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, or a mature storage medium in the art. The processor reads the information in the storage medium and completes the steps of the above method in combination with its hardware.

[0052] The storage medium may be a memory, which may be, for example, a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memory.

[0053] Among them, the non-volatile memory can be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory.

[0054] The volatile memory may be a random access memory (RAM) which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDRSDRAM), enhanced synchronous DRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus DRAM (DRRAM).

[0055] The storage media described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memory.

[0056] Those skilled in the art will appreciate that in one or more of the above examples, the functions described in the present invention can be implemented using a combination of hardware and software. When software is used, the corresponding functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. Storage media can be any available media that can be accessed by general or special-purpose computers.

[0057] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions and improvements made on the basis of the technical solution of the present invention should be included in the scope of protection of the present invention.

Claims

1. A municipal road state perception method based on sensor fusion, characterized in that: include: S1. Collect multimodal road information data from the perception network along the municipal roads, and process the road information data to generate standardized road feature vectors; S2. Based on the standardized road characteristic vector, the frequency domain characteristics of the road surface bearing capacity are extracted through time-frequency analysis, a coupling model of vehicle load and road surface response is established, and various road status indicators are calculated; S3, determine the influence weight of the damage factor of each road condition indicator, output the structural health index and mark the heat map of the key damage location; S4. Based on the heat map of key damage locations, the diffusion path of damage factors is modeled using a spatiotemporal graph convolutional network, and real-time traffic flow data and meteorological environmental parameters are integrated to establish an adaptive adjustment model for warning levels based on fuzzy reasoning. S5. When the risk entropy value output by the warning level adaptive adjustment model exceeds the dynamic threshold, a multi-level collaborative maintenance decision-making plan is triggered and a visualized road status digital twin is generated.

2. A municipal road state perception method based on sensor fusion according to claim 1, characterized in that: Collecting multimodal road information data from the sensing network along municipal roads includes the following sub-steps: The three-dimensional angular velocity and linear acceleration data are collected through the nine-axis inertial measurement unit, and the quaternion attitude solution algorithm is used to compensate for the point cloud coordinate offset caused by the vibration of the laser radar equipment, establish the rotation matrix from the carrier coordinate system to the world coordinate system, and calculate the Euler angle deviation between adjacent scan lines; When building a time synchronization model at the 5G edge node, a two-way timestamp exchange protocol is used to calibrate the clock deviation of each sensor. The spatial point cloud data of the lidar and the velocity vector information of the millimeter-wave radar are fused through an extended Kalman filter to generate a multimodal data set aligned in time and space.

3. The municipal road state perception method based on sensor fusion according to claim 1 is characterized in that: Calculate various road status indicators, including: construct a dual-channel neural network to process three-dimensional point cloud and infrared image based on standardized road feature vector, extract three-dimensional geometric features of cracks through the point cloud channel, calculate crack width, depth and direction parameters, including the following sub-steps: The point cloud channel uses a three-dimensional sparse convolutional network to extract crack features, voxelizes the original point cloud, calculates the variance of the normal vector distribution of each voxel, and captures the irregular geometric features of the crack edge through a deformable convolution kernel. When processing the infrared channel, an improved deep semantic segmentation network is used to segment the temperature anomaly area, and the heat conduction equation is combined to invert and calculate the internal defect depth of the material. A surface temperature field model considering the solar radiation absorption rate and the emissivity of the pavement material is established, and the implicit difference format of the three-dimensional heat conduction equation is solved.

4. A municipal road state perception method based on sensor fusion according to claim 3, characterized in that: When processing the infrared channel, an improved deep semantic segmentation network is used to segment the temperature anomaly area, and the depth of internal defects of the material is calculated by inversion of the heat conduction equation. A surface temperature field model considering the solar radiation absorption rate and the emissivity of the pavement material is established to solve the implicit difference format of the three-dimensional heat conduction equation, including the following sub-steps: Collect the time series temperature data of the infrared thermal imager and construct a boundary condition matrix including ambient temperature, wind speed and sunshine intensity; The conjugate gradient method is used to solve the inverse problem of unsteady-state heat conduction and iteratively optimize the distribution of equivalent thermal conductivity of internal defects in the material. The influence of temperature measurement noise on the inversion results is eliminated through regularization processing, and the isothermal gradient cloud map of the defect area is generated.

5. The municipal road state perception method based on sensor fusion according to claim 1 is characterized in that: Based on the structural health index and the heat map with key damage locations marked, the diffusion path of the damage factor is modeled using a spatiotemporal graph convolutional network, which integrates real-time traffic flow data and meteorological environment parameters to calculate the risk field intensity attenuation function, including the following sub-steps: Define the spatiotemporal propagation model of damage diffusion and establish the state transfer equation including crack length, traffic load frequency and temperature cycle number; The random walk algorithm is used to simulate the propagation path of damage factors in the road network and calculate the superposition intensity of the risk field of each road section; The fatigue characteristics of pavement materials are introduced to correct the attenuation coefficient, and an exponential attenuation model considering material aging is constructed.

6. A municipal road state perception method based on sensor fusion according to claim 5, characterized in that: Define the spatiotemporal propagation model of damage diffusion and establish the state transition equation including crack length, traffic load frequency and temperature cycle number, including the following sub-steps: Extract the correlation characteristics between crack growth rate and traffic flow in historical damage data and construct a hidden Markov state transition probability matrix; The precipitation and freeze-thaw cycle data from meteorological stations are integrated to calculate the acceleration factor of environmental factors on the state transition rate.

7. The municipal road state perception method based on sensor fusion according to claim 1 is characterized in that: A warning level adaptive adjustment model based on fuzzy reasoning is established. When the risk entropy value exceeds the dynamic threshold, a multi-level collaborative maintenance decision-making plan is triggered and a visualized road status digital twin is generated, including the following sub-steps: Four warning levels are divided according to the structural health index, and a dynamic threshold trigger mechanism is set: when the index is lower than the threshold for consecutive preset hours, the second-level response is initiated, and when two or more critical damages occur at the same time, the first-level response is triggered; A multi-agent simulation module is embedded in the digital twin to simulate the impact of different maintenance plans on traffic flow, establish a vehicle lane-changing model in the maintenance construction area, and calculate the capacity attenuation rate of the road section during the construction period.

8. A municipal road state perception system based on sensor fusion, characterized in that: include: Data collection and preprocessing module: collect multimodal road information data from the perception network along the municipal roads, process the road information data to generate standardized road feature vectors; Feature processing and damage assessment module: Based on the standardized road feature vector, the frequency domain characteristics of the road bearing capacity are extracted through time-frequency analysis, a coupling model of vehicle load and road response is established, and various road status indicators are calculated; the influence weight of the damage factor of each road status indicator is determined, and the structural health index is output and a heat map with key damage locations is marked; Decision support and visualization module: Based on the heat map of key damage locations, the diffusion path of damage factors is modeled using a spatiotemporal graph convolutional network, and real-time traffic flow data and meteorological environmental parameters are integrated to establish an adaptive adjustment model for warning levels based on fuzzy reasoning. When the risk entropy value output by the adaptive adjustment model of the warning level exceeds the dynamic threshold, a multi-level collaborative maintenance decision-making plan is triggered and a visualized road status digital twin is generated.

9. A computer storage medium, characterized in that: include: at least one memory and at least one processor; A memory for storing one or more program instructions; A processor, used to run one or more program instructions to execute a municipal road state perception method based on sensor fusion as described in any one of claims 1-7.

Images