A novel digital highway maintenance management method and system based on multi-source data

Abstract

This application relates to the technical field of highway maintenance management. Specifically, it discloses a new digital highway maintenance management method and system based on multi-source data. It uses image processing technology based on deep learning to perform multi-dimensional analysis on road crack images to identify crack types and quantify key parameters such as their width, length, and area. At the same time, by performing time-series analysis on the historical traffic flow data of the highway, it captures the time-series change trend and periodic pattern of road traffic flow. Furthermore, through a time-series derivation algorithm, it integrates the static crack detection results with the dynamic traffic load pattern to predict the expansion trend of cracks under future traffic loads, forming a long-term crack development prediction, so as to facilitate highway maintenance management based on the long-term prediction results of road cracks. Through the collaborative analysis of multi-dimensional data, this method can effectively achieve the accurate diagnosis and active intervention of highway conditions, thereby significantly improving the efficiency and effectiveness of highway maintenance.

Description

Technical Field

[0001] This application relates to the technical field of highway maintenance management, and more specifically, to a new digital highway maintenance management method and system based on multi-source data. Background Art

[0002] As an important part of the modern transportation network, the maintenance status of highways directly affects driving safety, traffic efficiency, and the public's satisfaction with traffic services. Traditional highway maintenance management mainly relies on manual inspections and experience-based judgments, suffering from problems such as low detection efficiency, insufficient quantification of diseases, and lagging maintenance decisions. With the development of computer vision technology, crack detection methods based on image recognition (such as edge detection and morphological segmentation) have achieved the preliminary positioning of cracks, providing a more intelligent technical means for highway maintenance management.

[0003] However, the traffic load on highways is one of the important factors affecting crack propagation. Most of the existing crack detection methods ignore the dynamic interaction relationship between traffic load and crack propagation, focusing more on the analysis of static crack image features and not considering the non-linear impact of dynamic changes in traffic load on the crack propagation rate, which may lead to the failure of preventive maintenance due to timing deviations.

[0004] Therefore, there is an expectation for a new digital highway maintenance management method and system based on multi-source data. Summary of the Invention

[0005] To solve the above technical problems, this application is proposed. The embodiments of this application provide a new digital highway maintenance management method and system based on multi-source data, which uses deep learning-based image processing technology to perform multi-dimensional analysis on road crack images to identify crack types and quantify key parameters such as their width, length, and area. At the same time, by performing time series analysis on the historical traffic flow data of highways, the time series change trend and periodic pattern of road traffic flow are captured. Furthermore, through a time series derivation algorithm, the static crack detection results are fused with the dynamic traffic load pattern to predict the propagation trend of cracks under future traffic loads, forming a long-term crack development prediction, so as to facilitate highway maintenance management based on the long-term prediction results of road cracks. Through the collaborative analysis of multi-dimensional data, this method can effectively achieve the accurate diagnosis and proactive intervention of highway conditions, thereby significantly improving the efficiency and effectiveness of highway maintenance.

[0006] According to one aspect of this application, a new digital highway maintenance management method based on multi-source data is provided, which includes:

[0007] Obtain multi-source data for highway maintenance, where the multi-source data for highway maintenance includes road crack images and historical traffic flow data;

[0008] Input the road crack image into a road crack recognizer based on a deep learning model to obtain a road crack recognition result, where the road crack recognition result includes crack type, width, length, and area;

[0009] Extract traffic flow time series features from the historical traffic flow data to obtain a road traffic flow time series pattern feature coding vector;

[0010] Perform time series derivation on the road crack recognition result based on the road traffic flow time series pattern feature coding vector to obtain a long-term prediction result of road cracks;

[0011] Generate highway maintenance management suggestions based on the long-term prediction result of road cracks.

[0012] According to another aspect of the present application, a new digital highway maintenance management system based on multi-source data is provided, which includes:

[0013] A highway maintenance multi-source data acquisition module for acquiring highway maintenance multi-source data, where the highway maintenance multi-source data includes road crack images and historical traffic flow data;

[0014] A road crack image recognition module for inputting the road crack image into a road crack recognizer based on a deep learning model to obtain a road crack recognition result, where the road crack recognition result includes crack type, width, length, and area;

[0015] A traffic flow data feature extraction module for extracting traffic flow time series features from the historical traffic flow data to obtain a road traffic flow time series pattern feature coding vector;

[0016] A time series derivation module for performing time series derivation on the road crack recognition result based on the road traffic flow time series pattern feature coding vector to obtain a long-term prediction result of road cracks;

[0017] A highway maintenance management suggestion generation module for generating highway maintenance management suggestions based on the long-term prediction result of road cracks.

[0018] Compared with the prior art, the novel digital highway maintenance management method and system based on multi-source data provided by this application utilize image processing technology based on deep learning to perform multi-dimensional analysis on road crack images to identify crack types and quantify key parameters such as their width, length, and area. At the same time, by performing time-series analysis on the historical traffic flow data of the highway, the time-series change trend and periodic pattern of the road traffic flow are captured. Furthermore, through a time-series derivation algorithm, the static crack detection results are fused with the dynamic traffic load pattern to predict the expansion trend of cracks under future traffic loads, forming a long-term crack development prediction to facilitate highway maintenance management based on the long-term prediction results of road cracks. Through the collaborative analysis of multi-dimensional data, this method can effectively achieve the accurate diagnosis and active intervention of highway conditions, thereby significantly improving the efficiency and effect of highway maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] By describing the embodiments of the present application in more detail with reference to the accompanying drawings, the above and other objects, features, and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation to the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0020] Figure 1 It is a flowchart of a novel digital highway maintenance management method based on multi-source data according to an embodiment of the present application.

[0021] Figure 2 It is a schematic diagram of data flow of a novel digital highway maintenance management method based on multi-source data according to an embodiment of the present application.

[0022] Figure 3 It is a flowchart of sub-step S4 of a novel digital highway maintenance management method based on multi-source data according to an embodiment of the present application.

[0023] Figure 4 It is a flowchart of sub-step S42 of a novel digital highway maintenance management method based on multi-source data according to an embodiment of the present application.

[0024] Figure 5 It is a flowchart of sub-step S423 of a novel digital highway maintenance management method based on multi-source data according to an embodiment of the present application.

[0025] Figure 6 It is a block diagram of a novel digital highway maintenance management system based on multi-source data according to an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] As shown in this application and the claims, unless the context clearly indicates otherwise, words such as "a", "an", "one", and / or "the" are not specifically singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0027] Although this application makes various references to certain modules in the system according to the embodiments of this application, however, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0028] Flowcharts are used in this application to illustrate the operations performed by the system according to the embodiments of this application. It should be understood that the operations before or below do not necessarily need to be executed precisely in order. On the contrary, various steps can be processed in reverse order or simultaneously as needed. At the same time, other operations can also be added to these processes, or one or several steps can be removed from these processes.

[0029] Next, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments of this application. It should be understood that this application is not limited by the exemplary embodiments described here.

[0030] It is worth noting that in this application, all actions of obtaining data are carried out on the premise of complying with the corresponding data protection regulations and policies of the country where it is located and obtaining the authorization given by the owner of the corresponding device.

[0031] To address the technical problems described in the above background art, this application proposes a new digital highway maintenance management method based on multi-source data. It uses image processing technology based on deep learning to perform multi-dimensional analysis on road crack images to identify crack types and quantify key parameters such as their width, length, and area. At the same time, by performing time series analysis on the historical traffic flow data of the highway, it captures the time series change trend and periodic pattern of the road traffic flow. Furthermore, through a time series derivation algorithm, it integrates the static crack detection results with the dynamic traffic load pattern to predict the expansion trend of cracks under future traffic loads, forming a long-term crack development prediction, so as to facilitate highway maintenance management based on the long-term prediction results of road cracks. Through the collaborative analysis of multi-dimensional data, this method can effectively achieve the accurate diagnosis and proactive intervention of the highway state, thus significantly improving the efficiency and effect of highway maintenance.

[0032] Figure 1It is a flowchart of a new digital highway maintenance management method based on multi-source data according to an embodiment of the present application. Figure 2 It is a schematic diagram of data flow of a new digital highway maintenance management method based on multi-source data according to an embodiment of the present application. As Figure 1 and Figure 2 shown, the new digital highway maintenance management method based on multi-source data includes the steps of: S1, obtaining multi-source data for highway maintenance, where the multi-source data for highway maintenance includes road crack images and historical traffic flow data; S2, inputting the road crack images into a road crack recognizer based on a deep learning model to obtain a road crack recognition result, where the road crack recognition result includes crack type, width, length, and area; S3, extracting traffic flow time series features from the historical traffic flow data to obtain a road traffic flow time series pattern feature coding vector; S4, performing time series derivation on the road crack recognition result based on the road traffic flow time series pattern feature coding vector to obtain a long-term prediction result of road cracks; S5, generating highway maintenance management suggestions based on the long-term prediction result of road cracks.

[0033] In the above new digital highway maintenance management method based on multi-source data, in step S1, obtaining multi-source data for highway maintenance, where the multi-source data for highway maintenance includes road crack images and historical traffic flow data. It should be understood that as an important transportation infrastructure, highway maintenance work is directly related to traffic safety, transportation efficiency, and service life. Road crack images, as an intuitive presentation of the road surface condition, can clearly show information such as the distribution, trend, and shape of cracks, and are the key basis for evaluating the current damage degree of the road. For example, through crack images, it can be judged whether the crack penetrates the road surface, which is crucial for evaluating the integrity of the road structure. Historical traffic flow data is closely related to the load borne by the road. The difference in traffic flow in different time periods and different road sections will lead to a significant difference in the load borne by the road, thereby affecting the damage process of the road. Therefore, by obtaining road crack images and comprehensive historical traffic flow data of the highway, the actual condition and usage of the road can be reflected from different angles, laying a solid data foundation for realizing precise highway maintenance management.

[0034] Specifically, in actual operation, road crack images are usually collected by means of a high-resolution camera or a drone. Through these devices, detailed information on the road surface, especially the specific location, shape, and distribution of cracks, can be captured. Selecting appropriate shooting time and weather conditions is crucial for ensuring image quality. A clear and cloudless day is conducive to reducing shadow interference, thereby obtaining clear and accurate crack images. In addition, in order to ensure data consistency and comparability, a unified standard needs to be formulated to regulate the image collection process, such as determining fixed shooting angles, distances, and coverage ranges, etc. This not only helps subsequent analysis and processing but also effectively improves the recognition accuracy.

[0035] Meanwhile, the collection of historical traffic flow data cannot be ignored. This type of data usually comes from the monitoring systems of traffic management departments, including but not limited to video surveillance, induction coils, license plate recognition systems, etc. Video surveillance can observe the traffic flow on the road in real time, but more importantly, the data accumulated over a long period can be used to analyze the traffic flow patterns at different times and in different seasons. Induction coils are buried under the road surface and generate signals whenever a vehicle passes by, thus counting the number of passing vehicles within a specific time period. This method is simple but very effective, especially suitable for traffic flow statistics on main roads or highways. The license plate recognition system calculates the traffic flow by recognizing the license plate numbers of passing vehicles and can further analyze additional information such as vehicle type distribution and average speed, providing a basis for in-depth study of the impact of traffic load on road damage.

[0036] To better understand the relationship between road cracks and traffic flow, it is necessary to conduct in-depth discussions with specific cases. For example, on a specific section of a highway, through long-term observation, it is found that due to its proximity to industrial areas and connection to important logistics centers, the traffic flow during the day, especially during the morning and evening rush hours, is extremely large, resulting in high pressure on the road surface. Correspondingly, some relatively serious crack problems have occurred on this section of the road. By comparing the road crack images at different times and combining with the historical traffic flow data analysis of the same period, it can be clearly seen that as the traffic flow increases, the crack propagation rate accelerates. This phenomenon indicates that traffic load is indeed one of the important factors affecting crack development.

[0037] In obtaining road crack images, special attention should also be paid to different types of cracks and their characteristic manifestations. According to characteristics such as crack width, length, and orientation, cracks can be divided into various types such as transverse cracks, longitudinal cracks, and reticulated cracks. Different types of cracks reflect different failure mechanisms and development trends. For example, transverse cracks are often related to stresses caused by temperature changes, while longitudinal cracks may be due to local damage caused by frequent vehicle rolling. Through detailed analysis of various crack images, not only can the current road damage degree be accurately judged, but also the possible problem areas in the future can be predicted, providing a basis for formulating a scientific and reasonable maintenance plan.

[0038] In the process of collecting historical traffic flow data, in addition to the regular vehicle number statistics, attention also needs to be paid to the impact of special events on traffic flow. For example, holidays, large-scale events, or emergencies may all lead to a sharp increase in traffic flow in the short term, thereby exacerbating the road burden. Therefore, in the data collation stage, the existence of these special situations should be fully considered and incorporated into the consideration scope. Only in this way can a complete traffic flow model that includes both daily regularities and abnormal fluctuations be constructed, laying a solid foundation for subsequent analysis.

[0039] In addition, considering that the environmental factors faced by different regions and different types of roads vary, corresponding adjustments need to be made according to specific circumstances during the data collection process. For example, in the northern regions with relatively harsh climate conditions, snow removal operations are frequent in winter, which not only causes additional damage to the road surface but also affects normal image acquisition work. At this time, it can be considered to increase the collection frequency or take special protective measures to ensure the normal operation of the equipment. In the rainy southern regions, attention should be paid to how the integrity of the drainage system affects the development of road cracks. Accordingly, the data collection plan should also have a focus, such as strengthening the monitoring of areas prone to water accumulation.

[0040] In the above new digital highway maintenance management method based on multi-source data, in step S2, the road crack image is input into a road crack recognizer based on a deep learning model to obtain a road crack recognition result, and the road crack recognition result includes crack type, width, length, and area. Specifically, since the traditional method of manually identifying road cracks is extremely inefficient, requires a large amount of manpower and time, and is prone to missed detections and misjudgments. Therefore, in order to improve the efficiency and accuracy of road crack recognition, this application uses a road crack recognizer based on a deep learning model to process the road crack image. In the embodiment of this application, a convolutional neural network (CNN) is used to construct the road crack recognizer, which is trained through a large number of labeled crack images and gradually learns the morphological, textural, and other characteristic patterns of different types of cracks. For example, transverse cracks have relatively regular horizontal line features, longitudinal cracks show vertical line features, and reticular cracks have complex reticular texture features. In the fully connected layer, the model integrates the extracted features and constructs a mapping relationship from the input image to the crack type through a non-linear activation function (such as ReLU), thereby realizing the automatic recognition of road cracks. Furthermore, by measuring the pixel size of the crack in the image and combining the known conversion relationship between pixels and actual sizes, the actual width, length, area, and other quantitative parameters of the crack can be obtained. In this way, the accurate recognition and quantitative analysis of cracks are effectively realized, which not only greatly improves the recognition efficiency but also reduces the missed detections and misjudgments caused by human factors, providing more reliable data support for subsequent highway maintenance management.

[0041] In the above-mentioned novel digital highway maintenance management method based on multi-source data, in step S3, traffic flow time-series features are extracted from the historical traffic flow data to obtain a road traffic flow time-series pattern feature coding vector. It should be understood that since road traffic flow has obvious time-series characteristics. For example, the traffic flow during morning and evening rush hours will increase sharply, and the load borne by the road will also increase correspondingly. The commuting demand on weekdays causes the traffic flow to be concentrated in specific time periods, while on weekends, it may show different distributions due to reasons such as leisure travel, etc. This is crucial for understanding the dynamic changes in the load borne by the road. Based on this, in the embodiments of the present application, a time-series encoder based on the LSTM model is used to extract time-series features from the historical traffic flow data to obtain the road traffic flow time-series pattern feature coding vector. LSTM (Long Short-Term Memory network) is a special type of recurrent neural network (RNN), and its core structure includes an input gate, a forget gate, and an output gate. Through these three gating units, LSTM can selectively retain historical information and forget irrelevant information, thereby effectively capturing long-term dependencies in time series. In the present application, when processing the historical traffic flow data, the data is sequentially input into the LSTM model in chronological order. The LSTM model saves historical information through its internal state and determines the output and updates the internal state based on the current input and historical state. In this way, the LSTM model can learn the time-series change trend and periodic law of road traffic flow, such as the difference in traffic flow between weekdays and weekends, the change in traffic flow during morning and evening rush hours, etc., and encode them into the road traffic flow time-series pattern feature coding vector, thus providing an important data basis for subsequent traffic load analysis and prediction.

[0042] In the above-mentioned novel digital highway maintenance management method based on multi-source data, in step S4, based on the road traffic flow time-series pattern feature coding vector, a time-series derivation is performed on the road crack identification result to obtain a long-term road crack prediction result. Specifically, since the development of road cracks is affected by the dynamic traffic load. As time goes by, the driving of vehicles causes repeated extrusion, stretching, and shearing effects on the cracks, resulting in the continuous expansion of the cracks. Moreover, this effect has a time cumulative effect, that is, under the long-term action of traffic load, the expansion speed and degree of the cracks will gradually increase. Therefore, in order to more accurately predict the expansion trend of cracks under future traffic load, the present application further performs a time-series derivation on the road crack identification result based on the road traffic flow time-series pattern feature coding vector, so as to combine static crack detection data with dynamic traffic load information, simulate the evolution process of cracks under traffic load, thereby helping highway maintenance management departments to formulate reasonable maintenance plans in advance, arrange maintenance resources reasonably, and further avoid high maintenance costs and traffic congestion caused by serious road damage due to crack development. Among them, Figure 3It is a flowchart of sub-step S4 of the novel digital highway maintenance management method based on multi-source data according to an embodiment of the present application. As Figure 3 shown, the step S4 includes steps: S41, performing structured mapping encoding on the road crack recognition result to obtain a structured embedded encoding vector of the current state of the road crack; S42, performing crack-traffic flow cross-modal depth interaction response encoding on the structured embedded encoding vector of the current state of the road crack and the traffic flow time-series pattern feature encoding vector to obtain a crack-traffic flow interaction response encoding vector; S43, decoding the crack-traffic flow interaction response encoding vector to obtain the long-term prediction result of the road crack.

[0043] Specifically, in the step S41, the road crack recognition result is subjected to structured mapping encoding to obtain a structured embedded encoding vector of the current state of the road crack. It should be understood that the present application takes into account that the road crack recognition result is presented in various attribute forms, such as crack type, width, length, and area. Therefore, in order to effectively fuse and analyze it with the traffic flow time-series pattern feature encoding vector with specific format and dimension requirements, the present application further performs structured mapping encoding on the road crack recognition result to integrate and transform the scattered crack attribute information into a unified and structured low-dimensional vector form. In the embodiment of the present application, first, for the discrete crack type attribute, the one-hot encoding method is adopted to convert it into a binary vector representation, so that each crack type has a unique and distinguishable representation in the vector space. For example, if there are three types of transverse cracks, longitudinal cracks, and reticular cracks, after one-hot encoding, the transverse crack may be represented as [1, 0, 0], the longitudinal crack as [0, 1, 0], and the reticular crack as [0, 0, 1]. For continuous numerical attributes such as crack width, length, and area, normalization processing is performed to map their value ranges to a unified interval, such as [0, 1], to eliminate the differences in numerical magnitudes of different attributes. Finally, after the preprocessed data is input into the fully connected neural network, the hidden layer in the network performs a linear operation on the input data through the weight matrix and introduces a non-linear factor through a non-linear activation function (such as the ReLU function) to enhance the network's ability to express complex relationships. After the transformation of multiple hidden layers, a low-dimensional vector containing multi-dimensional attribute information of the crack is finally output, that is, the structured embedded encoding vector of the current state of the road crack.

[0044] Specifically, in step S42, a cross-modal depth interaction response encoding of the current state structured embedding code vector of the road crack and the time series pattern feature encoding vector of the road traffic flow is performed to obtain a crack-traffic flow interaction response code vector. It should be understood that since the crack propagation process is not only determined by its initial form but also affected by the dynamic action of long-term traffic loads, and the time series characteristics of traffic flow itself have complex characteristics such as periodicity and volatility, it is difficult to capture their non-linear coupling relationship by simply linearly adding or simply splicing the two types of data. Therefore, in order to solve the problem of collaborative expression of two types of heterogeneous data and establish a dynamic association mechanism, the present application constructs an interaction architecture with soft constraints on modal independence, and on the basis of retaining the respective independence of the crack geometric features and traffic flow statistical features, adaptively mines the response correlation between the two in the time dimension (such as the acceleration effect of vehicle flow during peak hours on short-term crack propagation, the cumulative impact of seasonal vehicle flow changes on long-term crack development), so that the model can not only identify the static disease characteristics of cracks but also dynamically quantify the driving effect of traffic loads on their evolution process, thus significantly improving the spatio-temporal accuracy of crack propagation prediction. Among them, Figure 4 is a flowchart of sub-step S42 of the novel digital highway maintenance management method based on multi-source data according to an embodiment of the present application. As Figure 4 shown, step S42 includes the steps of: S421, performing principal component analysis on the time series pattern feature encoding vector of the road traffic flow to obtain a set of time series principal component feature encoding vectors of the road traffic flow; S422, performing fine-grained feature interaction response encoding on each time series principal component feature encoding vector in the set of the current state structured embedding code vector of the road crack and the set of time series principal component feature encoding vectors of the road traffic flow to obtain a set of fine-grained response interaction encoding vectors between the crack-time series principal component modalities; S423, based on the information independence between each time series principal component feature encoding vector in the set of the current state structured embedding code vector of the road crack and the set of time series principal component feature encoding vectors of the road traffic flow, adaptively aggregating the set of fine-grained response interaction encoding vectors between the crack-time series principal component modalities to obtain the crack-traffic flow interaction response code vector.

[0045] More specifically, step S421 is expressed by the formula:

[0046]

[0047] wherein, represents the time series pattern feature encoding vector of the road traffic flow, represents the principal component analysis function, represents the transpose of the vector, is The characteristic scale value, denotes the covariance matrix of which represents the matrix composed of the set of the principal component feature encoding vectors of the road traffic flow time series obtained by performing eigenvalue decomposition on the and denotes the diagonal matrix composed of the set of the principal component eigenvalues of the road traffic flow time series obtained by performing eigenvalue decomposition on the and , , denote each principal component eigenvalue of the road traffic flow time series in the set of the principal component eigenvalues of the road traffic flow time series, is the number of the principal component eigenvalues of the road traffic flow time series, , , denote each principal component feature encoding vector in the set of the principal component feature encoding vectors of the road traffic flow time series.

[0048] That is, since the traffic flow time series data itself has characteristics such as multi-dimension, strong correlation, and periodic fluctuations, directly inputting its original features into the interaction unit may cause the model to fall into the "curse of dimensionality", and at the same time, redundant features will interfere with the mining of the true relationship between cracks and traffic flow. Therefore, in order to solve the redundancy and information coupling problems of high-dimensional time series data, this application uses the principal component analysis, a feature space transformation method, to project the high-dimensional traffic flow time series data onto the low-dimensional orthogonal principal component axes, which not only greatly reduces the computational complexity, but more importantly, by eliminating the multicollinearity between features, retains the core variation information in the traffic flow time series data that can explain the largest variance, enabling the subsequent interaction process to focus on the dynamic traffic load patterns that play a leading role in crack propagation. This dimensionality reduction operation not only improves the computational efficiency of the model, but more importantly, through the information purification mechanism, strengthens the expression ability of key patterns such as periodicity and trend in the traffic flow time series features, thereby providing an efficient and refined feature representation basis for subsequent cross-modal interaction.

[0049] More specifically, in a specific example of this application, the step S422 includes: First, perform a linear transformation on each principal component feature encoding vector in the set of the principal component feature encoding vectors of the road traffic flow time series so that it has the same characteristic scale as the structured embedding encoding vector of the current state of the road crack to obtain a set of principal component linear transformation feature encoding vectors of the road traffic flow time series, which is expressed by the formula:

[0050]

[0051] where denotes the linear transformation function, Denote the set of linear transformation feature coding vectors of the principal components of the road traffic flow time series, , , respectively denote each linear transformation feature coding vector of the principal components of the road traffic flow time series in the set of linear transformation feature coding vectors of the principal components of the road traffic flow time series.

[0052] That is, although the core variation information has been extracted after the principal component analysis of the traffic flow time series data, there may still be significant differences in the original scales of its principal components. If directly interacting with the structured embedding coding vector of the crack state, it may lead to the coverage of high-dimensional features over low-dimensional features or the dilution of low-dimensional features in the interaction. Therefore, to solve the "information bias" problem caused by data dimension differences and numerical range imbalances in cross-modal feature interaction, this application performs scale transformation on the traffic flow principal components through linear transformation, which can not only eliminate the dimensional differences between different principal components, but more importantly, constructs a numerical benchmark matching the crack features, enabling the two types of heterogeneous features to have equal weight contribution capabilities in subsequent interactions.

[0053] Then, input the structured embedding coding vector of the current state of the road crack and each linear transformation feature coding vector of the principal components of the road traffic flow time series in the set of linear transformation feature coding vectors of the principal components of the road traffic flow time series into the feature interaction response unit to obtain the set of fine-grained response interaction coding vectors between the crack-traffic flow time series principal component modes, which is expressed by the formula:

[0054]

[0055] Among them, denotes the -th linear transformation feature coding vector of the principal components of the road traffic flow time series in the set of linear transformation feature coding vectors of the principal components of the road traffic flow time series, denotes dot product, denotes dot addition, denotes dot division, denotes and the fine-grained response interaction coding vector between the crack-traffic flow time series principal component modes, denotes vector concatenation, and respectively denote the weight matrix and bias term of the feature interaction response unit.

[0056] That is, since crack propagation is not only affected by its initial form, but also closely related to complex time series patterns such as the periodic fluctuations of traffic loads and sudden events (such as holiday peaks or the passage of heavy vehicles). Therefore, in order to deeply interactively mine the complementarity and dependence of two types of heterogeneous features, this application performs multi-level non-linear modeling on the current state features of cracks and the time series features of traffic flow through a feature interaction response unit, in order to capture the synergistic effects of crack geometric parameters and traffic flow dynamic patterns at the micro level (such as short-term load impacts in local areas) and the macro level (such as annual cycle load accumulation). For example, identifying the accelerating effect of sudden changes in traffic density on crack propagation rate during a specific time period or the superimposed effect of seasonal traffic flow changes on the development of long-term diseases. Thus, through fine-grained interaction response coding, a set of fine-grained response interaction coding vectors between the crack-traffic flow time series principal component modes is obtained, realizing a more accurate dynamic prediction of the crack propagation process.

[0057] Figure 5 It is a flowchart of sub-step S423 of the novel digital highway maintenance management method based on multi-source data according to an embodiment of the present application. As Figure 5 shown, the step S423 includes the steps of: S4231, inputting each road traffic flow time series principal component linear transformation feature coding vector in the set of the current state structured embedding coding vectors of the road crack and the road traffic flow time series principal component linear transformation feature coding vectors into an inter-modal independence modeling unit to obtain a set of crack-traffic flow time series principal component inter-modal independence coding matrices; S4232, calculating a set of crack-traffic flow time series principal component inter-modal independence soft constraint factors based on the set of crack-traffic flow time series principal component inter-modal independence coding matrices; S4233, dynamically and adaptively aggregating the set of crack-traffic flow time series principal component inter-modal fine-grained response interaction coding vectors based on the set of crack-traffic flow time series principal component inter-modal independence soft constraint factors to obtain the crack-traffic flow interaction response coding vector.

[0058] In a specific example of the present application, the step S4231 includes: inputting the current state structured embedding coding vector of the road crack and the road traffic flow time series principal component linear transformation feature coding vector into a non-linear activation function to obtain a road crack current state structured embedding activation coding vector and a road traffic flow time series principal component feature activation coding vector; calculating the product between the transposed vector of the road crack current state structured embedding activation coding vector and the road traffic flow time series principal component feature activation coding vector, and dividing the product result by the square root of the feature scale value of the road crack current state structured embedding coding vector to obtain the crack-traffic flow time series principal component inter-modal independence coding matrix. The above step S4231 is expressed by the formula:

[0059]

[0060] Among them, and are different non-linear activation functions, represents the structured embedding coding vector of the current state of the road crack, is 's characteristic scale value, is and the independence coding matrix between the crack-traffic flow time series principal component modes.

[0061] That is to say, since the traffic flow time series data, after principal component decomposition, the obtained set of principal component components not only contains the core patterns of the traffic flow time series changes, but also contains redundant information or implicit interference terms, which may mislead the model's accurate understanding of the relationship between crack propagation and traffic load. In this regard, the present application further constructs an inter-modal independence modeling unit to explicitly construct the independence and complementarity representation space between modes, quantify the degree of independence between crack features and traffic flow features, and strip redundant information through learnable constraint conditions, only retaining the complementary associations that actually contribute to the prediction of crack propagation, so that the model focuses more on the synergistic effect of the two types of features rather than repetitive information in subsequent interactions.

[0062] Specifically, in a preferred example of the present application, the step S4232 includes: First, perform topological structure stability optimization on the independence coding matrix between the crack-traffic flow time series principal component modes to obtain an optimized independence coding matrix between the crack-traffic flow time series principal component modes, which is expressed by the formula:

[0063]

[0064]

[0065]

[0066] Among them, , , respectively represent 's respective eigenvalues, is 's eigenvector, represents matrix multiplication operation, represents the first-order canonical independence coding vector between the crack-traffic flow time series principal component modes, represents the optimized independence coding matrix between the crack-traffic flow time series principal component modes.

[0067] Here, since the interaction process between the traffic flow time series characteristics and the crack geometric characteristics involves the coupling of local short-term load disturbances (such as the passage of sudden heavy-duty vehicles) and global long-term periodic patterns (such as annual traffic flow trends), traditional modeling methods are prone to local structural instability due to the asymmetry of the feature space, which in turn affects the global consistency of crack propagation prediction. To address the structural vulnerability problem caused by the asymmetric data distribution in cross-modal interaction and thus ensure the robust expression of the model under complex traffic environments and crack evolution dynamics. This application constructs a feature interaction framework with both local sensitivity and global stability. Through gauge potential optimization and topological invariant constraints, the eigenvector composed of the eigenvalues of the independence encoding matrix between the crack-traffic flow time series principal component modes is used as the eigen gauge potential representation under the topological order to maintain structural stability based on topological invariants in the asymmetric case. Specifically, after obtaining the eigenvector and ignoring the high-order terms in the eigenrepresentation far from the limit distribution, a feature vector is obtained based on the gauge potential - connection 1-form and the first-order gauge potential term . Then, for the associated phase accumulation of the associated topological structure under the gauge potential, the independence encoding matrix between the crack-traffic flow time series principal component modes is gauge-optimized in the form of the autocorrelation matrix of the feature vector to ensure the stability of the overall asymmetric system against local structural perturbations, so as to obtain a stable structural representation of the independence encoding matrix between the crack-traffic flow time series principal component modes , enabling it to accurately capture the non-linear coupling mechanism between crack propagation and traffic flow time series in the short-range (such as short-period fluctuations in local areas) and long-range (such as long-period trends across road sections) correlations.

[0068] Then, the square of the F-norm of the optimized independence encoding matrix between the crack-traffic flow time series principal component modes is calculated as the independence soft constraint factor between the crack-traffic flow time series principal component modes, which is expressed by the formula:

[0069]

[0070] where represents the square of the F-norm of the matrix, and represents the independence soft constraint factor between the crack-traffic flow time series principal component modes corresponding to .

[0071] That is, since the interaction relationship between crack propagation and traffic flow time series includes both long-term trends dominated by independence (such as the progressive impact of seasonal fluctuations in traffic flow on cracks) and short-term perturbations dominated by correlation (such as the instantaneous impact of sudden heavy traffic flows on cracks), traditional fusion strategies with rigid constraints are difficult to adapt to the dynamic requirements of complex scenarios. By introducing a soft constraint factor, it is possible to dynamically adjust the contribution ratio of the interaction response in the independence coding matrix with a flexible weight mechanism - when the independence between the main component of traffic flow and the crack state is relatively high, the attention to independence features is strengthened to reduce redundant interference; when the independence between the two is low, it is relatively relaxed, thus allowing the model to adaptively fuse complementary information to capture key driving factors.

[0072] In a specific example of this application, the step S4233 is expressed by the formula:

[0073]

[0074] Where is the normalized exponential function, represents the crack-traffic flow interaction response coding vector.

[0075] Here, the set of fine-grained response interaction coding vectors between the principal component modes of the crack-traffic flow time series generated previously already contains hierarchical association information between the crack state features and the principal component features of the traffic flow time series (such as the sensitivity of crack width to short-term traffic flow and the cumulative effect of long-term traffic flow trends on crack area). However, directly integrating all interaction information may lead to redundant superposition or key signal drowning. By introducing the set of independence soft constraint factors between the principal component modes of the crack-traffic flow time series, it is possible to dynamically adjust the weight contribution of each fine-grained response interaction coding vector between the principal component modes of the crack-traffic flow time series according to the information independence and complementarity between the crack state features and the principal component features of the traffic flow time series. That is, when the independence between a certain traffic flow principal component and the crack state is strong, it indicates that the two contain significant complementary information. At this time, the weight of this principal component in the aggregation process should be increased to fully explore its unique predictive value for crack propagation; on the contrary, when the independence between the two is weak, its weight is appropriately reduced to avoid interference from redundant information and ensure that the aggregated crack-traffic flow interaction response coding vector can accurately focus on the core factors affecting crack propagation. Through this interaction method, it helps to fully consider the coupling effect between traffic flow and crack morphology in the subsequent long-term prediction of road cracks, thereby achieving a more accurate prediction of the road crack propagation trend.

[0076] Specifically, in step S43, the crack-traffic flow interaction response coding vector is decoded to obtain the long-term road crack prediction result. It should be understood that in order to convert the complex coupling information extracted in the cross-modal interaction process into a quantifiable expression of the crack evolution law, based on the inverse mapping ability of deep learning, through a decoder structure (such as a fully connected layer structure), and using a non-linear activation function to gradually restore the spatio-temporal dimensions of the crack features, the crack geometric features and the traffic flow time-series dynamic patterns fused in the crack-traffic flow interaction response coding vector are remapped back to the original data space to generate physically meaningful crack expansion parameters (such as width growth rate, area change rate). In this way, the expansion rate and critical threshold of cracks under different traffic load conditions can be accurately quantified, enabling maintenance decisions to be actively intervened based on dynamic risk thresholds. At the same time, by eliminating the prediction bias caused by traditional methods ignoring the traffic-crack coupling effect, the generalization ability and engineering practicality of the prediction model are significantly improved.

[0077] In the above new digital highway maintenance management method based on multi-source data, in step S5, highway maintenance management suggestions are generated based on the long-term road crack prediction result. That is, according to the predicted development of cracks, combined with factors such as the importance of the road and traffic flow, corresponding maintenance management decision rules are formulated. For example, if it is predicted that the crack will rapidly expand in the short term and the traffic flow on the section is large, it is recommended to carry out repairs immediately; if the crack develops slowly, it can be appropriately arranged in the future maintenance plan.

[0078] Specifically, in specific implementation, when customizing maintenance management suggestions according to the long-term road crack prediction result, the uniqueness of each road needs to be fully understood. For example, on urban arterial roads or highways, due to the large traffic flow, the development of any crack may quickly evolve into a serious safety hazard. Therefore, for such traffic-intensive road sections, once the prediction shows that the crack will rapidly expand in the short term, immediate action should be taken to carry out repair work. This timely intervention can not only prevent the crack from further deteriorating and causing more serious structural damage, but also reduce the traffic interruption time caused by road repairs, ensuring the safety and convenience of public travel.

[0079] Meanwhile, for those road sections where crack development is relatively slow, the maintenance plan can be flexibly arranged according to specific circumstances. In such cases, although emergency treatment is not required, it is still necessary to closely monitor the changes in cracks and incorporate them into future regular inspection and maintenance schedules. This can not only effectively control costs but also ensure that necessary preventive maintenance measures can be carried out in a timely manner without affecting current traffic safety. In addition, for some road sections with low traffic flow or in remote areas, even if it is predicted that the cracks will expand, the time node of the repair operation can be appropriately postponed according to the actual situation, and resources can be preferentially allocated to more urgent areas to maximize the efficiency of resource utilization.

[0080] When formulating specific maintenance management suggestions, considerations should also be made in combination with the importance and functional orientation of the road. For example, for roads connecting important economic regions or tourist attractions, their maintenance status directly affects local economic development and social activities. Therefore, any crack problems on such roads should be highly regarded. Even if the crack expansion speed is slow, the repair work should be planned as early as possible to ensure that the road is always in the best condition. On the contrary, for some secondary roads or temporary access roads, if the prediction results show that the crack development has little impact on the overall traffic, a more relaxed maintenance strategy can be adopted, and more energy can be concentrated on key sections.

[0081] In summary, the novel digital highway maintenance management method based on multi-source data according to the embodiments of the present application is elucidated. It uses image processing technology based on deep learning to perform multi-dimensional analysis on road crack images to identify crack types and quantify key parameters such as their width, length, and area. At the same time, by performing time series analysis on the historical traffic flow data of the highway, the time series change trend and periodic law of the road traffic flow are captured. Furthermore, through the time series derivation algorithm, the static crack detection results are integrated with the dynamic traffic load pattern to predict the expansion trend of cracks under future traffic loads, forming a long-term crack development prediction, so as to facilitate highway maintenance management based on the long-term prediction results of road cracks. Through the collaborative analysis of multi-dimensional data, this method can effectively achieve the accurate diagnosis and active intervention of highway conditions, thereby significantly improving the efficiency and effect of highway maintenance.

[0082] Furthermore, a novel digital highway maintenance management system based on multi-source data is also provided.

[0083] Figure 6 It is a block diagram of the novel digital highway maintenance management system based on multi-source data according to the embodiments of the present application. As Figure 6As shown in the figure, the novel digital highway maintenance management system 100 based on multi-source data according to an embodiment of the present application includes: a highway maintenance multi-source data acquisition module 110 for acquiring highway maintenance multi-source data, where the highway maintenance multi-source data includes road crack images and historical traffic flow data; a road crack image recognition module 120 for inputting the road crack images into a road crack recognizer based on a deep learning model to obtain a road crack recognition result, where the road crack recognition result includes crack type, width, length, and area; a traffic flow data feature extraction module 130 for extracting traffic flow time series features from the historical traffic flow data to obtain a road traffic flow time series pattern feature coding vector; a time series derivation module 140 for performing time series derivation on the road crack recognition result based on the road traffic flow time series pattern feature coding vector to obtain a long-term road crack prediction result; and a highway maintenance management recommendation generation module 150 for generating highway maintenance management recommendations based on the long-term road crack prediction result.

[0084] Here, those skilled in the art can understand that the specific operations of each module in the above novel digital highway maintenance management system based on multi-source data have been introduced in detail in the description of the novel digital highway maintenance management method based on multi-source data above, and therefore, the repeated description thereof will be omitted. Figures 1 to 5 In the above embodiments, the descriptions of the various embodiments have their own emphases. For parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the unit division is only a logical function division, and there may be other division methods in actual implementation. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0085] The basic principles of the present invention have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, advantages, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, advantages, effects, etc. are essential for each embodiment of the present invention. In addition, the specific details of the above embodiments are only for the purpose of illustration and easy understanding, and are not limitations. The above details do not limit the present invention to necessarily adopt the above specific details to implement.

[0086]

[0087] ​For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be embraced by the present invention. Any reference signs in the claims should not be regarded as limiting the claimed claim involved.

[0088] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units stated in the system claims can also be implemented by one unit through software or hardware.

[0089] Finally, it should be noted that the above description has been given for purposes of illustration and description. In addition, the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A novel digital highway maintenance management method based on multi-source data, characterized in that Including: Obtain multi-source highway maintenance data, where the multi-source highway maintenance data includes road crack images and historical traffic flow data; Input the road crack images into a road crack recognizer based on a deep learning model to obtain a road crack recognition result, where the road crack recognition result includes crack type, width, length, and area; Extract traffic flow time-series features from the historical traffic flow data to obtain a road traffic flow time-series pattern feature encoding vector; Perform time-series derivation on the road crack recognition result based on the road traffic flow time-series pattern feature encoding vector to obtain a long-term road crack prediction result; Generate highway maintenance management suggestions based on the long-term road crack prediction result; Performing time-series derivation on the road crack recognition result based on the road traffic flow time-series pattern feature encoding vector to obtain a long-term road crack prediction result, including: Perform structured mapping encoding on the road crack recognition result to obtain a structured embedding encoding vector of the current state of the road crack; Perform crack-traffic flow cross-modal deep interaction response encoding on the structured embedding encoding vector of the current state of the road crack and the road traffic flow time-series pattern feature encoding vector to obtain a crack-traffic flow interaction response encoding vector; Decode the crack-traffic flow interaction response encoding vector to obtain the long-term road crack prediction result.

2. The novel digital highway maintenance management method based on multi-source data according to claim 1, characterized in that Extract traffic flow time-series features from the historical traffic flow data to obtain a road traffic flow time-series pattern feature encoding vector, including: Use a time-series encoder based on an LSTM model to perform time-series feature extraction on the historical traffic flow data to obtain the road traffic flow time-series pattern feature encoding vector.

3. The novel digital highway maintenance management method based on multi-source data according to claim 2, characterized in that, Performing crack-traffic flow cross-modal deep interaction response encoding on the structured embedding encoding vector of the current state of the road crack and the road traffic flow time-series pattern feature encoding vector to obtain a crack-traffic flow interaction response encoding vector, including: Perform principal component analysis on the road traffic flow time-series pattern feature encoding vector to obtain a set of road traffic flow time-series principal component feature encoding vectors; Perform fine-grained feature interaction response encoding on each road traffic flow time-series principal component feature encoding vector in the set of the structured embedding encoding vector of the current state of the road crack and the road traffic flow time-series principal component feature encoding vectors to obtain a set of crack-traffic flow time-series principal component modal fine-grained response interaction encoding vectors; Based on the information independence between each road traffic flow time-series principal component feature encoding vector in the set of the structured embedding encoding vector of the current state of the road crack and the road traffic flow time-series principal component feature encoding vectors, perform adaptive aggregation on the set of crack-traffic flow time-series principal component modal fine-grained response interaction encoding vectors to obtain the crack-traffic flow interaction response encoding vector.

4. The new digital highway maintenance management method based on multi-source data according to claim 3, wherein, Performing fine-grained feature interaction response encoding on each road traffic flow time-series principal component feature encoding vector in the set of the structured embedding encoding vector of the current state of the road crack and the road traffic flow time-series principal component feature encoding vectors to obtain a set of crack-traffic flow time-series principal component modal fine-grained response interaction encoding vectors, including: Perform a linear transformation on each of the road traffic flow time series principal component feature encoding vectors in the set of road traffic flow time series principal component feature encoding vectors so that it has the same feature scale as the current state structured embedding encoding vector of the road crack to obtain a set of road traffic flow time series principal component linear transformation feature encoding vectors; Input the current state structured embedding encoding vector of the road crack and each of the road traffic flow time series principal component linear transformation feature encoding vectors in the set of road traffic flow time series principal component linear transformation feature encoding vectors into the feature interaction response unit to obtain a set of fine-grained response interaction encoding vectors between the crack-traffic flow time series principal component modes.

5. The novel digital highway maintenance management method based on multi-source data according to claim 4, characterized in that, Based on the information independence between the current state structured embedding encoding vector of the road crack and each of the road traffic flow time series principal component feature encoding vectors in the set of road traffic flow time series principal component feature encoding vectors, perform adaptive aggregation on the set of fine-grained response interaction encoding vectors between the crack-traffic flow time series principal component modes to obtain the crack-traffic flow interaction response encoding vector, including: Input the current state structured embedding encoding vector of the road crack and each of the road traffic flow time series principal component linear transformation feature encoding vectors in the set of road traffic flow time series principal component linear transformation feature encoding vectors into the inter-modal independence modeling unit to obtain a set of crack-traffic flow time series principal component inter-modal independence encoding matrices; Based on the set of crack-traffic flow time series principal component inter-modal independence encoding matrices, calculate a set of crack-traffic flow time series principal component inter-modal independence soft constraint factors; Based on the set of crack-traffic flow time series principal component inter-modal independence soft constraint factors, perform dynamic adaptive aggregation on the set of fine-grained response interaction encoding vectors between the crack-traffic flow time series principal component modes to obtain the crack-traffic flow interaction response encoding vector.

6. The novel digital highway maintenance management method based on multi-source data according to claim 5, characterized in that, Input the current state structured embedding encoding vector of the road crack and each of the road traffic flow time series principal component linear transformation feature encoding vectors in the set of road traffic flow time series principal component linear transformation feature encoding vectors into the inter-modal independence modeling unit to obtain a set of crack-traffic flow time series principal component inter-modal independence encoding matrices, including: Input the current state structured embedding encoding vector of the road crack and the road traffic flow time series principal component linear transformation feature encoding vector into a non-linear activation function to obtain a current state structured embedding activation encoding vector of the road crack and a road traffic flow time series principal component feature activation encoding vector; Calculate the product between the transposed vector of the current state structured embedding activation encoding vector of the road crack and the road traffic flow time series principal component feature activation encoding vector, and divide the product result by the square root of the feature scale value of the current state structured embedding encoding vector of the road crack to obtain the crack-traffic flow time series principal component inter-modal independence encoding matrix.

7. The novel digital highway maintenance management method based on multi-source data according to claim 6, characterized in that, Based on the set of crack-traffic flow time series principal component inter-modal independence encoding matrices, calculate a set of crack-traffic flow time series principal component inter-modal independence soft constraint factors, including: Topologically optimize the independence coding matrix between the crack-traffic flow time series principal component modes to obtain an optimized independence coding matrix between the crack-traffic flow time series principal component modes; Calculate the square of the F-norm of the optimized independence coding matrix between the crack-traffic flow time series principal component modes as the independence soft constraint factor between the crack-traffic flow time series principal component modes.

8. A novel digital highway maintenance management system based on multi-source data, characterized in that, It includes: A highway maintenance multi-source data acquisition module for acquiring highway maintenance multi-source data, where the highway maintenance multi-source data includes road crack images and historical traffic flow data; A road crack image recognition module for inputting the road crack image into a road crack recognizer based on a deep learning model to obtain a road crack recognition result, where the road crack recognition result includes crack type, width, length, and area; A traffic flow data feature extraction module for extracting traffic flow time series features from the historical traffic flow data to obtain a road traffic flow time series pattern feature coding vector; A time series derivation module for performing time series derivation on the road crack recognition result based on the road traffic flow time series pattern feature coding vector to obtain a long-term road crack prediction result; A highway maintenance management recommendation generation module for generating highway maintenance management recommendations based on the long-term road crack prediction result; The time series derivation module is used for: Performing structured mapping coding on the road crack recognition result to obtain a structured embedding coding vector of the current state of the road crack; Performing crack-traffic flow cross-modal depth interaction response coding on the structured embedding coding vector of the current state of the road crack and the road traffic flow time series pattern feature coding vector to obtain a crack-traffic flow interaction response coding vector; Decoding the crack-traffic flow interaction response coding vector to obtain the long-term road crack prediction result.

Images