Novel digital road maintenance management method and system based on multi-source data

Abstract

The invention relates to the technical field of road maintenance management, and particularly discloses a novel digital road maintenance management method and system based on multi-source data, and the method comprises the steps: carrying out the multi-dimensional analysis of a road crack image through employing an image processing technology based on deep learning; the method comprises the following steps of: identifying a crack type and quantifying key parameters such as width, length and area of the crack, simultaneously, capturing a time sequence change trend and a periodic rule of road traffic flow by performing time sequence analysis on historical traffic flow data of a road, and further fusing a static crack detection result with a dynamic traffic load mode through a time sequence derivation algorithm to obtain a dynamic crack detection result. The expansion trend of the crack under the action of the future traffic load is predicted, and long-term crack development prediction is formed, so that road maintenance management is performed based on the long-term prediction result of the road crack. According to the method, accurate diagnosis and active intervention of the road state can be effectively realized through multi-dimensional data collaborative analysis, so that the efficiency and effect of road maintenance are remarkably improved.

Description

Technical Field

[0001] The present application relates to the technical field of highway maintenance management, and more specifically, to a novel digital highway maintenance management method and system based on multi-source data. Background Art

[0002] As an important part of the modern transportation network, the maintenance status of highways directly affects driving safety, traffic efficiency and public satisfaction with transportation services. Traditional highway maintenance management mainly relies on manual inspections and empirical judgments, which have problems such as low detection efficiency, insufficient quantification of defects, and delayed maintenance decisions. With the development of computer vision technology, crack detection methods based on image recognition (such as edge detection and morphological segmentation) have achieved preliminary positioning of cracks, providing a more intelligent technical means for highway maintenance management.

[0003] However, highway traffic load is one of the important factors affecting crack propagation. Most existing crack detection methods ignore the dynamic interaction between traffic load and crack propagation, and focus more on static crack image feature analysis without considering the nonlinear effect of dynamic changes in traffic load on crack propagation rate, which may lead to the failure of preventive maintenance due to timing deviation.

[0004] Therefore, a new digital highway maintenance management method and system based on multi-source data is expected. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a new digital highway maintenance management method and system based on multi-source data, which uses deep learning-based image processing technology to perform multi-dimensional analysis of road crack images to identify crack types and quantify key parameters such as width, length and area. At the same time, through time series analysis of historical traffic flow data of the highway, the time series change trend and periodic law of road traffic flow are captured. Then, the static crack detection results are integrated with the dynamic traffic load pattern through a time series deduction algorithm to predict the expansion trend of cracks under future traffic loads, forming a long-term crack development prediction, so as to facilitate highway maintenance management based on the long-term prediction results of road cracks. This method can effectively realize accurate diagnosis and active intervention of highway status through collaborative analysis of multi-dimensional data, thereby significantly improving the efficiency and effectiveness of highway maintenance.

[0006] According to one aspect of the present application, a new digital highway maintenance management method based on multi-source data is provided, which includes: Obtain multi-source data on highway maintenance, including road crack images and historical traffic flow data; Inputting the road crack image into a road crack identifier based on a deep learning model to obtain a road crack identification result, wherein the road crack identification result includes crack type, width, length and area; Extracting traffic flow time series characteristics from the historical traffic flow data to obtain a road traffic flow time series pattern characteristic coding vector; Based on the road traffic flow time series pattern feature coding vector, the road crack identification result is deduced in time series to obtain a long-term prediction result of the road crack; Based on the long-term prediction results of road cracks, highway maintenance management recommendations are generated.

[0007] According to another aspect of the present application, a new digital highway maintenance management system based on multi-source data is provided, which includes: A highway maintenance multi-source data acquisition module is used to acquire highway maintenance multi-source data, which includes road crack images and historical vehicle flow data; A road crack image recognition module, used for inputting the road crack image into a road crack identifier based on a deep learning model to obtain a road crack recognition result, wherein the road crack recognition result includes crack type, width, length and area; A vehicle flow data feature extraction module, used to extract vehicle flow time series features from the historical vehicle flow data to obtain a road vehicle flow time series pattern feature encoding vector; A time series derivation module, used for performing time series derivation on the road crack identification result based on the road traffic flow time series pattern feature coding vector to obtain a long-term prediction result of the road crack; The highway maintenance management suggestion generation module is used to generate highway maintenance management suggestions based on the long-term prediction results of road cracks.

[0008] Compared with the prior art, the new digital highway maintenance management method and system based on multi-source data provided by this application uses deep learning-based image processing technology to perform multi-dimensional analysis of road crack images to identify crack types and quantify key parameters such as width, length and area. At the same time, by performing time series analysis on historical traffic flow data of the highway, the time series change trend and periodic law of road traffic flow are captured. Then, the static crack detection results are integrated with the dynamic traffic load pattern through a time series derivation algorithm to predict the expansion trend of cracks under future traffic loads, forming a long-term crack development prediction, so as to facilitate highway maintenance management based on the long-term prediction results of road cracks. This method can effectively realize accurate diagnosis and active intervention of highway status through multi-dimensional data collaborative analysis, thereby significantly improving the efficiency and effectiveness of highway maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0010] Figure 1 This is a flow chart of a new digital highway maintenance management method based on multi-source data according to an embodiment of the present application.

[0011] Figure 2 This is a data flow diagram of a new digital highway maintenance management method based on multi-source data according to an embodiment of the present application.

[0012] Figure 3 This is a flowchart of sub-step S4 of the novel digital highway maintenance management method based on multi-source data according to an embodiment of the present application.

[0013] Figure 4 This is a flowchart of sub-step S42 of the novel digital highway maintenance management method based on multi-source data according to an embodiment of the present application.

[0014] Figure 5 This is a flowchart of sub-step S423 of the novel digital highway maintenance management method based on multi-source data according to an embodiment of the present application.

[0015] Figure 6 It is a block diagram of a new digital highway maintenance management system based on multi-source data according to an embodiment of the present application. DETAILED DESCRIPTION

[0016] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0017] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0018] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0019] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0020] It is worth noting that in this application, all actions to obtain data are carried out in compliance with the relevant data protection laws and policies of the country where the data is located, and with the authorization given by the owner of the corresponding device.

[0021] In response to the technical problems described in the above background technology, this application proposes a new digital highway maintenance management method based on multi-source data, which uses deep learning-based image processing technology to perform multi-dimensional analysis of road crack images to identify crack types and quantify key parameters such as width, length and area. At the same time, through time series analysis of historical traffic flow data of the highway, the time series change trend and periodic law of road traffic flow are captured. Then, the static crack detection results are integrated with the dynamic traffic load pattern through a time series deduction algorithm to predict the expansion trend of cracks under future traffic loads, forming a long-term crack development prediction, so as to facilitate highway maintenance management based on the long-term prediction results of road cracks. This method can effectively realize accurate diagnosis and active intervention of highway status through multi-dimensional data collaborative analysis, thereby significantly improving the efficiency and effectiveness of highway maintenance.

[0022] Figure 1 This is a flow chart of a new digital highway maintenance management method based on multi-source data according to an embodiment of the present application. Figure 2 Schematic diagram of data flow of a novel digital highway maintenance management method based on multi-source data according to an embodiment of the present application. Figure 1 and Figure 2As shown, the novel digital highway maintenance management method based on multi-source data includes the following steps: S1, acquiring multi-source highway maintenance data, the multi-source highway maintenance data including road crack images and historical vehicle flow data; S2, inputting the road crack images into a road crack identifier based on a deep learning model to obtain a road crack identification result, the road crack identification result including crack type, width, length and area; S3, extracting vehicle flow time series features from the historical vehicle flow data to obtain a road vehicle flow time series pattern feature coding vector; S4, performing time series deduction on the road crack identification result based on the road vehicle flow time series pattern feature coding vector to obtain a road crack long-term prediction result; S5, generating a highway maintenance management recommendation based on the road crack long-term prediction result.

[0023] In the above-mentioned novel digital highway maintenance management method based on multi-source data, the step S1 obtains highway maintenance multi-source data, which includes road crack images and historical traffic flow data. It should be understood that as an important transportation infrastructure, the maintenance of highways is directly related to traffic safety, transportation efficiency and service life. As an intuitive presentation of the road surface condition, the road crack image can clearly display information such as the distribution, direction and morphology of the cracks, and is a key basis for evaluating the current degree of damage to the road. For example, the crack image can be used to determine whether the crack runs through the road surface, which is crucial for evaluating the integrity of the road structure. The historical traffic flow data is closely related to the load borne by the road. The difference in traffic flow in different time periods and different sections will lead to significant differences in the load borne by the road, thereby affecting the damage process of the road. Therefore, by obtaining the road crack image and comprehensive historical traffic flow data of the highway, the actual condition and use of the road can be reflected from different angles, laying a solid data foundation for achieving accurate highway maintenance management.

[0024] Specifically, in actual operation, the collection of road crack images is usually carried out with the help of high-resolution cameras or drones. Through these devices, detailed information on the road surface can be captured, especially the specific location, shape and distribution of the cracks. Choosing the appropriate shooting time and weather conditions is crucial to ensuring image quality. Clear and cloudless days are conducive to reducing shadow interference, thereby obtaining clear and accurate crack images. In addition, in order to ensure the consistency and comparability of data, unified standards need to be formulated to regulate the image acquisition process, such as determining fixed shooting angles, distances, and coverage. This will not only help with subsequent analysis and processing, but also effectively improve recognition accuracy.

[0025] At the same time, the collection of historical traffic data cannot be ignored. This type of data usually comes from the monitoring system of the traffic management department, including but not limited to video surveillance, induction coils, license plate recognition systems, etc. Video surveillance can observe the traffic flow on the road in real time, but more importantly, the data accumulated over a long period of time can be used to analyze the traffic flow patterns in different time periods and seasons. The induction coil is buried under the road surface, and a signal is generated every time a vehicle passes by, thereby counting the number of vehicles passing through in a specific time period. Although this method is simple, it is very effective and is particularly suitable for traffic flow statistics on main roads or highways. The license plate recognition system calculates the flow by identifying the license plate numbers of passing vehicles, and can further analyze additional information such as vehicle model distribution and average speed, providing a basis for in-depth research on the impact of traffic load on road damage.

[0026] In order to better understand the relationship between road cracks and traffic volume, it is necessary to conduct in-depth discussions based on specific cases. For example, on a specific section of a highway, after long-term observation, it was found that the road was close to an industrial area and connected to an important logistics center. The traffic volume during the day, especially during peak hours in the morning and evening, was extremely large, resulting in high pressure on the road surface. Accordingly, some serious crack problems appeared on this section of the road. By comparing the road crack images of different periods and combining the historical traffic volume data analysis of the same period, it can be clearly seen that as the traffic volume increases, the crack expansion rate tends to accelerate. This phenomenon shows that traffic load is indeed one of the important factors affecting the development of cracks.

[0027] In acquiring road crack images, special attention should be paid to the different types of cracks and their characteristic manifestations. According to the characteristics of crack width, length and direction, cracks can be divided into multiple types such as transverse cracks, longitudinal cracks, and mesh cracks. Different types of cracks reflect different damage mechanisms and development trends. For example, transverse cracks are often related to stress caused by temperature changes, while longitudinal cracks may be local damage caused by frequent vehicle rolling. Through the careful analysis of various types of crack images, not only can the current degree of road damage be accurately judged, but also possible problem areas in the future can be predicted, providing a basis for formulating scientific and reasonable maintenance plans.

[0028] In the process of collecting historical traffic flow data, in addition to the conventional vehicle count, it is also necessary to pay attention to the impact of special events on traffic flow. For example, holidays, large-scale events or emergencies may lead to a sharp increase in traffic flow in the short term, thereby increasing the burden on roads. Therefore, in the data collation stage, the existence of these special circumstances should be fully considered and included in the scope of consideration. Only in this way can a complete traffic flow model that includes both daily rules and abnormal fluctuations be constructed, laying a solid foundation for subsequent analysis.

[0029] In addition, considering that different regions and types of roads face different environmental factors, corresponding adjustments need to be made according to specific circumstances during the data collection process. For example, in the northern region with harsh climatic conditions, snow removal operations are frequent in winter, which will not only cause additional damage to the road surface, but also affect normal image collection. At this time, you can consider increasing the acquisition frequency or taking special protective measures to ensure the normal operation of the equipment. In the rainy areas in the south, we should pay attention to how the integrity of the drainage system affects the development of road cracks. Accordingly, the data collection plan should also focus on some aspects, such as strengthening the monitoring of areas prone to waterlogging.

[0030] In the above-mentioned novel digital highway maintenance management method based on multi-source data, in step S2, the road crack image is input into a road crack identifier based on a deep learning model to obtain a road crack identification result, and the road crack identification result includes crack type, width, length and area. Specifically, the traditional manual road crack identification method is extremely inefficient, requires a lot of manpower and time, and is prone to missed detection and misjudgment. Therefore, in order to improve the efficiency and accuracy of road crack identification, the present application uses a road crack identifier based on a deep learning model to process the road crack image. In an embodiment of the present application, a convolutional neural network (CNN) is used to construct a road crack identifier, which is trained by a large number of annotated crack images and gradually learns the morphology, texture and other characteristic patterns of different types of cracks. For example, transverse cracks have relatively regular transverse line features, longitudinal cracks present vertical line features, and mesh cracks have complex mesh texture features. In the fully connected layer, the model integrates the extracted features and constructs a mapping relationship from the input image to the crack type through a nonlinear activation function (such as ReLU), thereby realizing automatic recognition of road cracks. Furthermore, by measuring the pixel size of the crack in the image and combining the known conversion relationship between pixels and actual size, quantitative parameters such as the actual width, length and area of ​​the crack can be obtained. In this way, accurate identification and quantitative analysis of cracks are effectively achieved, which not only greatly improves the efficiency of identification, but also reduces missed detection and misjudgment caused by human factors, providing more reliable data support for subsequent highway maintenance management.

[0031] In the above-mentioned novel digital highway maintenance management method based on multi-source data, the step S3 extracts the time series characteristics of traffic flow from the historical traffic flow data to obtain the road traffic flow time series pattern feature encoding vector. It should be understood that since the road traffic flow has obvious time series characteristics, for example, the traffic flow in the morning and evening peak hours will increase sharply, and the load on the road will also increase accordingly. The commuting demand on weekdays causes the traffic flow to be more concentrated in a specific time period, while the weekend may show different distributions due to leisure travel and other reasons, which is very important for understanding the dynamic changes of the load on the road. Based on this, in an embodiment of the present application, a time series encoder based on the LSTM model is used to extract the time series characteristics of the historical traffic flow data to obtain the road traffic flow time series pattern feature encoding vector. LSTM (Long Short-Term Memory Network) is a special recurrent neural network (RNN), whose core structure includes an input gate, a forget gate and an output gate. Through these three gate control units, LSTM can selectively retain historical information and forget irrelevant information, thereby effectively capturing long-term dependencies in time series. In this application, when processing the historical traffic flow data, the data is input into the LSTM model in chronological order. The LSTM model saves historical information through its internal state, and determines the output and updates the internal state according to the current input and historical state. In this way, the LSTM model can learn the time series change trend and periodic law of road traffic flow, such as the difference in traffic flow on weekdays and weekends, the change in traffic flow during morning and evening peak hours, etc., and encode it into a road traffic flow time series pattern feature encoding vector, thereby providing an important data basis for subsequent traffic load analysis and prediction.

[0032] In the above-mentioned new digital highway maintenance and management method based on multi-source data, the step S4 performs time-series deduction on the road crack identification results based on the road traffic flow time-series pattern feature coding vector to obtain long-term prediction results for road cracks. Specifically, the development of road cracks is dynamically affected by traffic loads. As time goes by, the driving of vehicles produces repeated squeezing, stretching and shearing effects on the cracks, causing the cracks to continue to expand. Moreover, this influence has a time accumulation effect, that is, the cracks will gradually increase in expansion speed and degree under the long-term traffic load. Therefore, in order to more accurately predict the expansion trend of cracks under future traffic loads, the present application further performs time-series deduction on the road crack identification results based on the road traffic flow time-series pattern feature coding vector, so as to combine static crack detection data with dynamic traffic load information, and simulate the evolution process of cracks under traffic loads, thereby helping highway maintenance management departments to formulate reasonable maintenance plans in advance and arrange maintenance resources reasonably, thereby avoiding problems such as high maintenance costs and traffic congestion caused by severe damage to roads due to crack development. Among them, Figure 3FIG. 4 is a flowchart of sub-step S4 of the novel digital highway maintenance management method based on multi-source data according to an embodiment of the present application. Figure 3 As shown, the step S4 includes the steps of: S41, performing structured mapping encoding on the road crack identification result to obtain a structured embedded coding vector of the current state of the road crack; S42, performing crack-traffic flow cross-modal deep interactive response encoding on the structured embedded coding vector of the current state of the road crack and the road traffic flow time series pattern feature coding vector to obtain a crack-traffic flow interactive response coding vector; S43, decoding the crack-traffic flow interactive response coding vector to obtain the long-term prediction result of the road crack.

[0033] Specifically, the step S41 performs structured mapping encoding on the road crack identification result to obtain a structured embedded coding vector of the current state of the road crack. It should be understood that the present application takes into account that the road crack identification result is presented in the form of multiple attributes, such as crack type, width, length and area, so in order to effectively integrate and analyze it with the road traffic flow time series pattern feature coding vector with specific format and dimension requirements, the present application further performs structured mapping encoding on the road crack identification result to integrate and convert the scattered crack attribute information into a unified, structured low-dimensional vector form. In an embodiment of the present application, first, for the discrete crack type attribute, a one-hot encoding method is used to convert it into a binary vector representation, so that each crack type has a unique and distinguishable representation in the vector space. For example, if there are three types of transverse cracks, longitudinal cracks, and mesh cracks, after one-hot encoding, the transverse cracks may be represented as [1,0,0], the longitudinal cracks as [0,1,0], and the mesh cracks as [0,0,1]. For continuous numerical attributes such as crack width, length and area, normalization is performed to map their value range to a unified interval, such as [0,1], eliminating the differences in numerical magnitudes of different attributes. Finally, after the preprocessed data is input into the fully connected neural network, the hidden layer in the network performs linear operations with the input data through the weight matrix, and introduces nonlinear factors through nonlinear activation functions (such as ReLU functions) to enhance the network's ability to express complex relationships. After the transformation of multiple hidden layers, a low-dimensional vector containing multi-dimensional attribute information of the crack is finally output, that is, the structured embedded coding vector of the current state of the road crack.

[0034] Specifically, the step S42 performs crack-traffic flow cross-modal deep interactive response encoding on the structured embedded coding vector of the current state of the road crack and the road traffic flow time series pattern feature coding vector to obtain a crack-traffic flow interactive response coding vector. It should be understood that since the expansion process of the crack is not only determined by its initial form, but also affected by the long-term dynamic effect of traffic load, and the traffic flow time series characteristics themselves have complex characteristics such as periodicity and volatility, it is difficult to capture their nonlinear coupling relationship by simply linearly superimposing or simply splicing the two types of data. Therefore, in order to solve the problem of collaborative expression of two types of heterogeneous data and establish a dynamic association mechanism, the present application constructs an interactive architecture with soft constraints on inter-modal independence. On the basis of retaining the independence of crack geometric features and traffic flow statistical features, it adaptively mines the response correlation between the two in the time dimension (such as the acceleration effect of peak-hour traffic on short-term crack expansion, and the cumulative impact of seasonal traffic changes on long-term crack development), so that the model can not only identify the static disease characteristics of the crack, but also dynamically quantify the driving effect of traffic load on its evolution process, thereby significantly improving the spatiotemporal accuracy of crack expansion prediction. Among them, Figure 4 FIG. 4 is a flowchart of sub-step S42 of the novel digital highway maintenance management method based on multi-source data according to an embodiment of the present application. Figure 4 As shown, the step S42 includes the steps of: S421, performing principal component analysis on the road traffic flow time series pattern feature coding vector to obtain a set of road traffic flow time series principal component feature coding vectors; S422, performing fine-grained feature interactive response coding on each road traffic flow time series principal component feature coding vector in the set of the road crack current state structured embedded coding vector and the road traffic flow time series principal component feature coding vector to obtain a set of fine-grained response interactive coding vectors between crack-traffic flow time series principal component modes; S423, based on the information independence between the road crack current state structured embedded coding vector and each road traffic flow time series principal component feature coding vector in the set of the road traffic flow time series principal component feature coding vector, adaptively aggregating the set of fine-grained response interactive coding vectors between crack-traffic flow time series principal component modes to obtain the crack-traffic flow interactive response coding vector.

[0035] More specifically, the step S421 is expressed by the formula: in, represents the characteristic coding vector of the road traffic flow temporal pattern, represents the principal component analysis function, represents the transpose of a vector, for The characteristic scale value of express The covariance matrix of Indicates that by The matrix composed of the set of principal component feature coding vectors of road traffic flow time series obtained by eigenvalue decomposition is Indicates that by The diagonal matrix composed of the set of principal component eigenvalues ​​of the road traffic flow time series obtained by eigenvalue decomposition is , , represents each road traffic flow time series principal component eigenvalue in the set of road traffic flow time series principal component eigenvalues, is the number of principal component eigenvalues ​​of road traffic flow time series, , , Represents each road traffic flow time series principal component feature coding vector in a set of road traffic flow time series principal component feature coding vectors.

[0036] That is, since the traffic flow time series data itself has the characteristics of multi-dimensionality, strong correlation and periodic fluctuation, directly inputting its original features into the interaction unit may cause the model to fall into the "dimensional disaster", and the redundant features will interfere with the mining of the real relationship between cracks and traffic flow. Therefore, in order to solve the redundancy and information coupling problems of high-dimensional time series data, this application projects the high-dimensional traffic flow time series data onto the low-dimensional orthogonal principal component axis through the feature space transformation method of principal component analysis, which not only greatly reduces the computational complexity, but more importantly, by eliminating the multicollinearity between features, it retains the core variation information in the traffic flow time series data that can explain the maximum variance, so that the subsequent interaction process focuses on the dynamic mode of traffic load that plays a leading role in crack expansion. This dimensionality reduction operation not only improves the computational efficiency of the model, but more importantly, through the information purification mechanism, it strengthens the expression ability of key modes such as periodicity and trend in the traffic flow time series characteristics, thereby providing an efficient and refined feature representation basis for subsequent cross-modal interaction.

[0037] More specifically, in a specific example of the present application, the step S422 includes: first, linearly transforming each road traffic flow time series principal component feature coding vector in the set of the road traffic flow time series principal component feature coding vectors so that it has the same characteristic scale as the road crack current state structured embedded coding vector to obtain a set of road traffic flow time series principal component linear transformation feature coding vectors, which is expressed as: in, represents the linear transformation function, Represents the set of linear transformation feature encoding vectors of the principal components of road traffic flow time series, , , Respectively represent each road traffic flow time series principal component linear transformation feature coding vector in the set of road traffic flow time series principal component linear transformation feature coding vectors.

[0038] That is, although the core variation information has been extracted from the vehicle flow time series data after principal component analysis, the original scales of the principal components may still be significantly different. If they interact directly with the structured embedded coding vector of the crack state, it may cause high-dimensional features to cover low-dimensional features or low-dimensional features to be diluted in the interaction. Therefore, in order to solve the "information bias" problem caused by data dimension differences and numerical range imbalance in cross-modal feature interaction, this application scales the principal components of vehicle flow through linear transformation, which can not only eliminate the dimensional differences between different principal components, but more importantly, construct a numerical benchmark that matches the crack characteristics, so that the two types of heterogeneous features have equal weight contribution capabilities in subsequent interactions.

[0039] Then, each road traffic flow time series principal component linear transformation feature coding vector in the set of the road crack current state structured embedding coding vector and the road traffic flow time series principal component linear transformation feature coding vector is input into the feature interactive response unit to obtain the set of fine-grained response interactive coding vectors between the crack and traffic flow time series principal component modes, which is expressed as follows: in, The first one in the set of principal component linear transformation feature coding vectors of the road traffic flow time series is represented The principal component linear transformation feature encoding vector of the road traffic flow time series, represents dot product, Indicates point addition, Indicates point division, express and The cracks between the traffic flow time series and the fine-grained response interaction encoding vector between the principal component modes, represents vector concatenation, and They respectively represent the weight matrix and bias term of the feature interaction response unit.

[0040] That is, since crack expansion is not only affected by its initial form, but also closely related to complex time series patterns such as periodic fluctuations of traffic loads, sudden events (such as holiday peaks or heavy-loaded vehicles), etc. Therefore, in order to deeply and interactively explore the complementarity and dependence of the two types of heterogeneous features, this application uses feature interactive response units to perform multi-level nonlinear modeling of the current state characteristics of the cracks and the time series characteristics of the traffic flow, so as to capture the synergistic effects of crack geometric parameters and traffic flow dynamic patterns at the micro (such as short-term load impacts in local areas) and macro (such as annual cycle load accumulation) levels, such as identifying the acceleration of the crack expansion rate due to sudden changes in traffic density in a specific time period or the superimposed effects of seasonal traffic changes on the development of long-term diseases, so as to obtain a collection of fine-grained response interactive coding vectors between the main component modes of the crack-traffic flow time series through fine-grained interactive response coding, so as to achieve a more accurate dynamic prediction of the crack expansion process.

[0041] Figure 5 FIG. 4 is a flowchart of sub-step S423 of the novel digital highway maintenance management method based on multi-source data according to an embodiment of the present application. Figure 5 As shown, the step S423 includes the steps of: S4231, inputting the structured embedding coding vector of the current state of the road crack and each road traffic flow time series principal component linear transformation feature coding vector in the set of the road traffic flow time series principal component linear transformation feature coding vector into the inter-modal independence modeling unit to obtain a set of crack-traffic flow time series principal component modal inter-independence coding matrices; S4232, calculating a set of crack-traffic flow time series principal component modal inter-independence soft constraint factors based on the set of crack-traffic flow time series principal component modal inter-independence coding matrices; S4233, based on the set of crack-traffic flow time series principal component modal inter-independence soft constraint factors, dynamically and adaptively aggregating the set of crack-traffic flow time series principal component modal fine-grained response interaction coding vectors to obtain the crack-traffic flow interaction response coding vector.

[0042] In a specific example of the present application, the step S4231 includes: inputting the road crack current state structured embedding coding vector and the road traffic flow time series principal component linear transformation feature coding vector into a nonlinear activation function to obtain the road crack current state structured embedding activation coding vector and the road traffic flow time series principal component feature activation coding vector; calculating the product between the transposed vector of the road crack current state structured embedding activation coding vector and the road traffic flow time series principal component feature activation coding vector, and dividing the product result by the square root of the characteristic scale value of the road crack current state structured embedding coding vector to obtain the crack-traffic flow time series principal component modal independence coding matrix. The above step S4231 is expressed by the formula: in, and For different nonlinear activation functions, Represents the current state of road cracks structured embedding encoding vector, for The characteristic scale value of for and The independence coding matrix between the principal components of the crack-traffic flow time series.

[0043] That is, after the traffic flow time series data is decomposed by principal components, the principal component set obtained not only contains the core pattern of traffic flow time series changes, but also contains redundant information or implicit interference items, which may mislead the model's accurate understanding of the relationship between crack expansion and traffic load. In this regard, the present application further constructs an inter-modal independence modeling unit to explicitly construct the inter-modal independence and complementarity representation space, quantify the degree of independence between crack characteristics and traffic flow characteristics, and remove redundant information through learnable constraints, retaining only complementary associations that have a practical contribution to crack expansion prediction, so that the model can focus more on the synergistic effect of the two types of features rather than repeated information in subsequent interactions.

[0044] In particular, in a preferred example of the present application, the step S4232 includes: first, performing topological structure stability optimization on the crack-traffic flow time series principal component modal independence coding matrix to obtain an optimized crack-traffic flow time series principal component modal independence coding matrix, which is expressed as: in, , , Respectively Each eigenvalue of for The eigenvectors of represents the matrix multiplication operation, represents the first-order canonical crack-traffic flow time series principal component modal independence coding vector, Represents the independence coding matrix between the principal component modes of the optimized crack-traffic flow time series.

[0045] Here, since the interaction process between the time series characteristics of traffic flow and the geometric characteristics of cracks involves the coupling of local short-term load disturbances (such as sudden heavy-load vehicle traffic) and global long-term periodic laws (such as annual traffic trends), traditional modeling methods are prone to local structural instability due to the asymmetry of the feature space, thereby affecting the global consistency of crack extension prediction. In order to deal with the structural vulnerability problems caused by asymmetric data distribution in cross-modal interactions, and thus ensure the robust expression of the model under complex traffic environments and crack evolution dynamics. The present application constructs a feature interaction framework that has both local sensitivity and global stability. Through canonical potential optimization and topological invariant constraints, the eigenvector composed of the eigenvalues ​​of the independence encoding matrix between the principal component modes of the crack-traffic flow time series is used as the eigennormal potential representation under the topological order to maintain structural stability based on topological invariants in asymmetric cases. Specifically, after obtaining the eigenvector After that, ignoring the higher-order terms in the case of the eigenrepresentation far from the limiting distribution, the first-order gauge potential term is obtained based on the gauge potential-connection 1 form The eigenvector of Then, for the correlation phase accumulation of the correlation topological structure under the gauge potential, the eigenvector The independence encoding matrix between the principal component modes of the crack-traffic flow time series is obtained by using the autocorrelation matrix form The specification optimization is performed to ensure the stability of the overall asymmetric system to local structural disturbances, so as to obtain the independence coding matrix between the principal component modes of the crack-traffic flow time series The stable structural representation of the crack propagation can accurately capture the nonlinear coupling mechanism between crack propagation and traffic flow time series in both short-range (such as short-period fluctuations in local areas) and long-range (such as long-period trends across road sections).

[0046] Then, the square of the F norm of the optimization crack-traffic flow time series principal component modal independence coding matrix is ​​calculated as the soft constraint factor of the crack-traffic flow time series principal component modal independence, which is expressed by the formula: in, represents the square of the F-norm of the matrix, express The corresponding soft constraint factor for independence between the principal component modes of the crack-traffic flow time series.

[0047] That is, since the interactive relationship between crack expansion and traffic flow time series includes both long-term trends dominated by independence (such as the gradual impact of seasonal fluctuations in traffic flow on cracks) and short-term disturbances dominated by correlation (such as the instantaneous impact of sudden heavy traffic on cracks), the traditional rigid constraint fusion strategy is difficult to adapt to the dynamic needs of complex scenarios. By introducing soft constraint factors, the contribution ratio of the interactive response in the independence coding matrix can be dynamically adjusted with a flexible weight mechanism - when the independence of the main components of traffic flow and the crack state is high, the focus on independent features is strengthened to reduce redundant interference; when the independence of the two is low, it is relatively relaxed, allowing the model to adaptively fuse complementary information to capture key driving factors.

[0048] In a specific example of the present application, the step S4233 is expressed by the formula: in, is the normalized exponential function, Represents the crack-traffic flow interaction response encoding vector.

[0049] Here, the set of fine-grained response interaction coding vectors between the crack-traffic flow time series principal component modes generated in the early stage already contains the hierarchical correlation information between the crack state characteristics and the traffic flow time series principal component characteristics (such as the sensitivity of the crack width to short-term traffic flow, the cumulative effect of the long-term traffic flow trend on the crack area), but directly integrating all the interaction information may lead to redundant superposition or key signal drowning. By introducing the set of independent soft constraint factors between the crack-traffic flow time series principal component modes, the weight contribution of each crack-traffic flow time series principal component mode fine-grained response interaction coding vector can be dynamically adjusted according to the information independence and complementarity between the crack state characteristics and the traffic flow time series principal component characteristics. That is, when the independence of a certain traffic flow principal component and the crack state is strong, it indicates that the two contain significant complementary information. At this time, the weight of the principal component in the aggregation process should be enhanced to fully explore its unique predictive value for crack expansion; on the contrary, when the independence of the two is weak, their weights should be appropriately reduced to avoid redundant information interference, so as to ensure that the aggregated crack-traffic flow interaction response coding vector can accurately focus on the core factors affecting crack expansion. This interactive method helps to fully consider the coupling effect of traffic flow and crack morphology in the subsequent long-term prediction of road cracks, thereby achieving a more accurate prediction of the expansion trend of road cracks.

[0050] Specifically, the step S43 decodes the crack-traffic flow interactive response encoding vector to obtain the long-term prediction result of the road crack. It should be understood that in order to convert the complex coupling information extracted in the cross-modal interaction process into a quantifiable expression of crack evolution law, the present application is based on the inverse mapping capability of deep learning, through a decoder structure (such as a fully connected layer structure), using a nonlinear activation function to gradually restore the spatiotemporal dimensions of crack characteristics, and remap the crack geometric features and the traffic flow time series dynamic mode fused in the crack-traffic flow interactive response encoding vector back to the original data space, and generate crack extension parameters with physical significance (such as width growth rate, area change rate). In this way, the expansion rate and critical threshold of cracks under different traffic load conditions can be accurately quantified, so that maintenance decisions can be actively intervened based on dynamic risk thresholds. At the same time, by eliminating the prediction bias caused by ignoring the traffic-crack coupling effect of traditional methods, the generalization ability and engineering practicality of the prediction model are significantly improved.

[0051] In the above-mentioned novel digital highway maintenance management method based on multi-source data, the step S5 generates highway maintenance management suggestions based on the long-term prediction results of road cracks. That is, according to the predicted development of cracks, combined with factors such as the importance of the road and traffic flow, corresponding maintenance management decision rules are formulated. For example, if it is predicted that the cracks will expand rapidly in the short term and the traffic flow of the road section is large, it is recommended to repair them immediately; if the cracks develop slowly, they can be appropriately arranged in the future maintenance plan.

[0052] Specifically, in practice, when customizing maintenance and management recommendations based on long-term prediction results of road cracks, it is necessary to fully understand the uniqueness of each road. For example, on urban main roads or highways, due to the heavy traffic volume, the development of any cracks may quickly evolve into a serious safety hazard. Therefore, for such busy road sections, once the prediction shows that the cracks will expand rapidly in the short term, immediate action should be taken to repair them. This timely intervention can not only prevent the cracks from further deteriorating and causing more serious structural damage, but also reduce the traffic interruption time caused by road maintenance, ensuring the safety and convenience of public travel.

[0053] At the same time, for those sections of roads where cracks develop relatively slowly, maintenance plans can be flexibly arranged according to specific circumstances. In this case, although there is no need for emergency treatment, it is still necessary to pay close attention to the changes in the cracks and include them in the future regular inspection and maintenance schedule. This can not only effectively control costs, but also ensure that necessary preventive maintenance measures are carried out in a timely manner without affecting current traffic safety. In addition, for some sections with low traffic volume or located in remote areas, even if it is predicted that the cracks will expand, the time node of the repair operation can be appropriately postponed according to the actual situation, and resources can be allocated to more urgent areas first, so as to maximize the benefits of resource utilization.

[0054] When formulating specific maintenance and management recommendations, it is also necessary to consider the importance and functional positioning of the road. For example, the maintenance status of roads connecting important economic areas or tourist attractions directly affects local economic development and social activities. Therefore, any crack problems on such roads should be given high attention. Even if the cracks expand slowly, repair work should be planned as early as possible to ensure that the road is always in the best condition. On the contrary, for some minor roads or temporary passages, if the prediction results show that the development of cracks has little impact on overall traffic, a more relaxed maintenance strategy can be adopted to focus more on key sections.

[0055] In summary, a new digital highway maintenance management method based on multi-source data based on the embodiment of the present application is explained, which uses deep learning-based image processing technology to perform multi-dimensional analysis on road crack images to identify crack types and quantify key parameters such as width, length and area. At the same time, by performing time series analysis on historical traffic flow data of the highway, the time series change trend and periodic law of road traffic flow are captured. Then, the static crack detection results are integrated with the dynamic traffic load pattern through the time series deduction algorithm to predict the expansion trend of cracks under future traffic loads, forming a long-term crack development prediction, so as to facilitate highway maintenance management based on the long-term prediction results of road cracks. This method can effectively realize accurate diagnosis and active intervention of highway status through multi-dimensional data collaborative analysis, thereby significantly improving the efficiency and effectiveness of highway maintenance.

[0056] Furthermore, a new digital highway maintenance management system based on multi-source data is also provided.

[0057] Figure 6 FIG. 1 is a block diagram of a novel digital highway maintenance management system based on multi-source data according to an embodiment of the present application. Figure 6As shown, according to the embodiment of the present application, the novel digital highway maintenance management system 100 based on multi-source data includes: a highway maintenance multi-source data acquisition module 110, which is used to acquire highway maintenance multi-source data, and the highway maintenance multi-source data includes road crack images and historical vehicle flow data; a road crack image recognition module 120, which is used to input the road crack image into a road crack identifier based on a deep learning model to obtain a road crack recognition result, and the road crack recognition result includes crack type, width, length and area; a vehicle flow data feature extraction module 130, which is used to extract vehicle flow time series features from the historical vehicle flow data to obtain a road vehicle flow time series pattern feature coding vector; a time series derivation module 140, which is used to perform time series derivation on the road crack recognition result based on the road vehicle flow time series pattern feature coding vector to obtain a road crack long-term prediction result; a highway maintenance management suggestion generation module 150, which is used to generate a highway maintenance management suggestion based on the road crack long-term prediction result.

[0058] Here, those skilled in the art can understand that the specific operations of each module in the novel digital highway maintenance management system based on multi-source data have been described in the above reference. Figures 1 to 5 The novel digital highway maintenance management method based on multi-source data has been introduced in detail, and therefore, its repeated description will be omitted.

[0059] The basic principle of the present invention is described above in conjunction with specific embodiments. However, it should be pointed out that the advantages, strengths, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present invention. In addition, the specific details of the above embodiments are only for the purpose of illustration and facilitation of understanding, rather than limitation, and the above details do not limit the present invention to being implemented by adopting the above specific details.

[0060] In the above embodiments, the description of each embodiment has its own emphasis. For the parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiment described above is only schematic. For example, the unit division is only a logical function division, and there may be other division methods in actual implementation. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0061] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference to a figure in a claim should not be considered as limiting the claim to which it relates.

[0062] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units stated in the system claims can also be implemented by one unit through software or hardware.

[0063] Finally, it should be noted that the above description has been given for the purpose of illustration and description. In addition, the above embodiments are only used to illustrate the technical solution of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A new digital highway maintenance management method based on multi-source data, characterized in that: include: Obtain multi-source data on highway maintenance, including road crack images and historical traffic flow data; Inputting the road crack image into a road crack identifier based on a deep learning model to obtain a road crack identification result, wherein the road crack identification result includes crack type, width, length and area; Extracting traffic flow time series characteristics from the historical traffic flow data to obtain a road traffic flow time series pattern characteristic coding vector; Based on the road traffic flow time series pattern feature coding vector, the road crack identification result is deduced in time series to obtain a long-term prediction result of the road crack; Based on the long-term prediction results of road cracks, highway maintenance management recommendations are generated.

2. The novel digital highway maintenance management method based on multi-source data according to claim 1 is characterized in that: Extracting the traffic flow time series characteristics from the historical traffic flow data to obtain the road traffic flow time series pattern characteristic coding vector includes: A time series encoder based on the LSTM model is used to extract time series features from the historical vehicle flow data to obtain a feature encoding vector of the road vehicle flow time series pattern.

3. The novel digital highway maintenance management method based on multi-source data according to claim 2 is characterized in that: The road crack identification result is deduced in time series based on the road traffic flow time series pattern feature coding vector to obtain a long-term prediction result of the road crack, including: Performing structured mapping coding on the road crack identification result to obtain a structured embedded coding vector of the current state of the road crack; Performing crack-vehicle flow cross-modal deep interactive response encoding on the road crack current state structured embedding coding vector and the road vehicle flow temporal pattern feature coding vector to obtain a crack-vehicle flow interactive response coding vector; The crack-traffic flow interaction response encoding vector is decoded to obtain the road crack long-term prediction result.

4. The novel digital highway maintenance management method based on multi-source data according to claim 3 is characterized in that: Performing crack-vehicle flow cross-modal deep interactive response coding on the road crack current state structured embedded coding vector and the road vehicle flow temporal pattern feature coding vector to obtain a crack-vehicle flow interactive response coding vector, including: Performing principal component analysis on the road traffic flow time series pattern feature coding vector to obtain a set of road traffic flow time series principal component feature coding vectors; Performing fine-grained feature interactive response coding on each road traffic flow time series principal component feature coding vector in the set of the road crack current state structured embedded coding vector and the road traffic flow time series principal component feature coding vector to obtain a set of fine-grained response interactive coding vectors between crack-traffic flow time series principal component modes; Based on the information independence between the current state structured embedded coding vector of the road crack and each road traffic flow time series principal component feature coding vector in the set of the road traffic flow time series principal component feature coding vectors, the set of fine-grained response interaction coding vectors between the crack-traffic flow time series principal component modes is adaptively aggregated to obtain the crack-traffic flow interaction response coding vector.

5. The novel digital highway maintenance management method based on multi-source data according to claim 4 is characterized in that: Performing fine-grained feature interactive response coding on each road traffic flow time series principal component feature coding vector in the set of the road crack current state structured embedded coding vector and the road traffic flow time series principal component feature coding vector to obtain a set of fine-grained response interactive coding vectors between crack-traffic flow time series principal component modes, including: Performing a linear transformation on each road traffic flow time series principal component feature coding vector in the set of road traffic flow time series principal component feature coding vectors so that the vectors have the same feature scale as the road crack current state structured embedded coding vector to obtain a set of road traffic flow time series principal component linear transformation feature coding vectors; The current state of the road crack is structurally embedded in the coding vector and each road traffic flow time series principal component linear transformation feature coding vector in the set of the road traffic flow time series principal component linear transformation feature coding vector is input into the feature interaction response unit to obtain a set of fine-grained response interaction coding vectors between the crack and traffic flow time series principal component modes.

6. The novel digital highway maintenance management method based on multi-source data according to claim 5 is characterized in that: Based on the information independence between the road crack current state structured embedded coding vector and each road traffic flow time series principal component feature coding vector in the set of the road traffic flow time series principal component feature coding vectors, adaptively aggregating the set of fine-grained response interaction coding vectors between the crack and traffic flow time series principal component modalities to obtain the crack-traffic flow interaction response coding vector, including: Inputting each road traffic flow time series principal component linear transformation feature coding vector in the set of the road crack current state structured embedding coding vector and the road traffic flow time series principal component linear transformation feature coding vector into the inter-modal independence modeling unit to obtain a set of crack-traffic flow time series principal component inter-modal independence coding matrices; Based on the set of independence coding matrices between the main component modes of the crack-traffic flow time series, a set of independence soft constraint factors between the main component modes of the crack-traffic flow time series is calculated; Based on the set of independence soft constraint factors between the main component modes of the crack-traffic flow time series, the set of fine-grained response interaction coding vectors between the main component modes of the crack-traffic flow time series is dynamically and adaptively aggregated to obtain the crack-traffic flow interaction response coding vector.

7. The novel digital highway maintenance management method based on multi-source data according to claim 6 is characterized in that: Inputting each road traffic flow time series principal component linear transformation feature coding vector in the set of the road crack current state structured embedding coding vector and the road traffic flow time series principal component linear transformation feature coding vector into the inter-modal independence modeling unit to obtain a set of crack-traffic flow time series principal component inter-modal independence coding matrices, including: Inputting the road crack current state structured embedding coding vector and the road traffic flow time series principal component linear transformation feature coding vector into a nonlinear activation function to obtain the road crack current state structured embedding activation coding vector and the road traffic flow time series principal component feature activation coding vector; The product of the transposed vector of the structured embedding activation coding vector of the current state of the road cracks and the feature activation coding vector of the principal component of the road traffic flow time series is calculated, and the product result is divided by the square root of the feature scale value of the structured embedding coding vector of the current state of the road cracks to obtain the independence coding matrix between the principal component modes of the crack-traffic flow time series.

8. The novel digital highway maintenance management method based on multi-source data according to claim 7 is characterized in that: Based on the set of independence coding matrices between the main component modes of the crack-traffic flow time series, a set of independence soft constraint factors between the main component modes of the crack-traffic flow time series is calculated, including: Performing topological structure stability optimization on the crack-traffic flow time series principal component modal independence coding matrix to obtain an optimized crack-traffic flow time series principal component modal independence coding matrix; The square of the F-norm of the optimized crack-traffic flow time series principal component modal independence coding matrix is ​​calculated as the soft constraint factor of the crack-traffic flow time series principal component modal independence.

9. A new digital highway maintenance management system based on multi-source data, characterized in that: include: A highway maintenance multi-source data acquisition module is used to acquire highway maintenance multi-source data, which includes road crack images and historical vehicle flow data; A road crack image recognition module, used for inputting the road crack image into a road crack identifier based on a deep learning model to obtain a road crack recognition result, wherein the road crack recognition result includes crack type, width, length and area; A vehicle flow data feature extraction module, used to extract vehicle flow time series features from the historical vehicle flow data to obtain a road vehicle flow time series pattern feature encoding vector; A time series derivation module, used for performing time series derivation on the road crack identification result based on the road traffic flow time series pattern feature coding vector to obtain a long-term prediction result of the road crack; The highway maintenance management suggestion generation module is used to generate highway maintenance management suggestions based on the long-term prediction results of road cracks.

Images