Highway road condition monitoring method and system based on multi-dimensional data fusion

Abstract

The present application provides a highway road condition monitoring method and system based on multi-dimensional data fusion, which first collects a learning data sequence containing multi-dimensional road condition data and its detailed feature annotations (including road condition feature labels and severity). Subsequently, by encoding these data, a multi-dimensional coding vector sequence containing road condition state trends, feature labels and severity is generated. Using these coding vectors, the method trains a highway road condition prediction network by performing two network learning tasks: one is to predict road condition feature labels based on road condition state trends and feature label encoding; the other is to predict road condition severity based on road condition state trends and severity encoding. Through these two tasks, the network can learn the deep features of the road condition data, and then realize the accurate road condition features and severity prediction of any newly collected multi-dimensional road condition data, effectively improving the accuracy and real-time performance of highway road condition monitoring.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and more specifically, to a highway road condition monitoring method and system based on multi-dimensional data fusion. Background Art

[0002] In the field of transportation, as an important part of modern transportation, real-time monitoring and prediction of highway road conditions are of great significance for improving traffic management efficiency and ensuring driving safety. Traditional highway road condition monitoring methods often rely on a single data source, such as traffic flow counters, fixed cameras or manual inspections. Although these methods can provide road condition information to a certain extent, they have shortcomings such as single data dimension, poor real-time performance and limited coverage. Especially in the face of complex and changing traffic environments, a single data source is often difficult to fully and accurately reflect the actual road conditions. Summary of the invention

[0003] In view of the above-mentioned problems, in combination with the first aspect of the present application, an embodiment of the present application provides a highway road condition monitoring method based on multi-dimensional data fusion, the method comprising:

[0004] Acquire a first highway road condition learning data sequence, wherein the highway road condition learning data in the first highway road condition learning data sequence includes sample multi-dimensional acquisition fusion data and road condition feature annotation data of the sample multi-dimensional acquisition fusion data, wherein the road condition feature annotation data includes a road condition feature label and a road condition severity of the sample multi-dimensional acquisition fusion data; the road condition feature annotation data of at least one of the highway road condition learning data in the first highway road condition learning data sequence includes a plurality of road condition feature labels;

[0005] Encode and represent each of the highway road condition learning data to generate a coding vector sequence for each of the highway road condition learning data, wherein the coding vector sequence for the highway road condition learning data includes a sample road condition state trend coding vector of the sample multi-dimensional acquisition and fusion data, a road condition feature label coding vector of the sample multi-dimensional acquisition and fusion data, and a road condition severity coding vector of the sample multi-dimensional acquisition and fusion data;

[0006] According to the sample road condition state trend coding vector and the road condition feature label coding vector of the sample multi-dimensional acquisition and fusion data, a first network learning task of a highway road condition prediction network is performed, wherein the first network learning task is used to perform parameter learning on the highway road condition prediction network so that the highway road condition prediction network predicts the road condition feature label corresponding to the sample multi-dimensional acquisition and fusion data;

[0007] Based on the sample road condition state trend coding vector and the road condition severity coding vector of the sample multi-dimensional acquisition and fusion data, the second network learning task of the highway road condition prediction network is run, and the second network learning task is used to perform parameter learning on the highway road condition prediction network so that the highway road condition prediction network predicts the road condition severity of the sample multi-dimensional acquisition and fusion data, so that the highway road condition prediction network that has completed parameter learning predicts the road condition feature label and road condition severity of any target multi-dimensional acquisition and fusion data.

[0008] On the other hand, an embodiment of the present application also provides a cloud computing system, including a processor and a machine-readable storage medium, wherein the machine-readable storage medium is connected to the processor, the machine-readable storage medium is used to store programs, instructions or codes, and the processor is used to run the programs, instructions or codes in the machine-readable storage medium to implement the above method.

[0009] Based on the above aspects, the embodiment of the present application significantly improves the accuracy and real-time performance of highway road condition monitoring by integrating multi-dimensional road condition data and performing deep learning and fusion processing. Specifically, by introducing multi-dimensional annotation data of road condition feature labels and road condition severity, combined with the fine coding of sample road condition state trends, the complex characteristics of road condition changes can be captured more comprehensively, thereby greatly improving the accuracy of road condition prediction. In addition, the embodiment of the present application is not limited to the monitoring of specific sections or time periods, but the highway road condition prediction network trained by a large amount of learning data has a strong generalization ability and can effectively predict the road conditions of different highways and different time periods. The multi-dimensional collected data (such as traffic flow, speed, weather conditions, etc.) is fused and processed to form a comprehensive road condition status representation, which overcomes the limitation of incomplete information of a single data source and provides a richer and more accurate data basis for road condition monitoring. The entire monitoring process is highly automated, and real-time monitoring and prediction of road conditions can be achieved without human intervention, which significantly improves traffic management efficiency and reduces labor costs. In addition, for complex road conditions, multiple road condition feature labels can be predicted simultaneously, not limited to a single state description, providing traffic management departments with more detailed road condition analysis reports, which helps to formulate more refined traffic management strategies. In other words, through multi-dimensional data fusion and deep learning methods, the comprehensiveness, accuracy and real-time monitoring of highway road conditions are achieved, providing strong technical support for improving traffic management level and public travel experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a schematic diagram of the execution flow of the highway road condition monitoring method based on multi-dimensional data fusion provided in an embodiment of the present application.

[0011] Figure 2It is a schematic diagram of the hardware architecture of the cloud computing system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0012] The present application will be described in detail below with reference to the accompanying drawings. Figure 1 It is a flow chart of a highway road condition monitoring method based on multi-dimensional data fusion provided by an embodiment of the present application. The highway road condition monitoring method based on multi-dimensional data fusion is introduced in detail below.

[0013] Step S110, obtaining a first highway road condition learning data sequence, wherein the highway road condition learning data in the first highway road condition learning data sequence includes sample multi-dimensional acquisition fusion data and road condition feature annotation data of the sample multi-dimensional acquisition fusion data, wherein the road condition feature annotation data includes a road condition feature label and road condition severity of the sample multi-dimensional acquisition fusion data. The road condition feature annotation data of at least one of the highway road condition learning data in the first highway road condition learning data sequence includes a plurality of road condition feature labels.

[0014] In detail, the first highway road condition learning data sequence is an ordered data set, which includes multiple highway road condition learning data items, and is used to train or learn a highway road condition prediction model. Specifically, it can be collected from an actual highway monitoring and traffic management system for subsequent data analysis and model training. The highway road condition learning data is the basic unit that constitutes the first highway road condition learning data sequence. Each data item includes multi-dimensional road condition information about a certain period or a certain section of the highway, as well as the annotation of this information (i.e., road condition feature annotation data).

[0015] The sample multi-dimensional acquisition fusion data is the raw data collected by various sensors distributed along the highway (such as cameras, speedometers, weather stations, etc.) and the database of the traffic management department. After processing, these data are fused into multi-dimensional road condition information, including but not limited to traffic flow, speed, vehicle type, weather conditions, road construction conditions, accident information, etc.

[0016] The road condition feature annotation data is a detailed annotation of the sample multi-dimensional collection and fusion data, including road condition feature labels and road condition severity, which is used to guide the model to learn how to extract useful road condition information from the original data and predict future road conditions.

[0017] The road condition feature label is an abstract representation of the road condition feature, which is used to describe a certain state or feature of the road condition. For example, "congestion during peak hours", "slow traffic caused by bad weather", etc. A road condition feature label may contain multiple road condition feature knowledge points, which are further detailed descriptions of the road condition features. The road condition severity is a quantitative representation of the severity of the road condition, usually divided into several levels, such as "mild", "moderate", "severe", etc. It is used to evaluate the impact of road conditions on traffic flow.

[0018] Therefore, in this embodiment, the server obtains a first highway road condition learning data sequence, which is collected from various sensors distributed along the highway (such as cameras, speed meters, weather stations, etc.) and the database of the traffic management department.

[0019] The sample multi-dimensional acquisition fusion data may include information from multiple aspects, such as the traffic volume, speed distribution, vehicle type ratio, weather conditions (such as sunny, rainy, foggy, etc.), road construction conditions, location and type of accidents, etc. on a specific road section within a certain period of time.

[0020] The road condition feature labels in the road condition feature annotation data can be very complex and diverse. For example, the road condition feature label of "congestion during peak hours" may not only include the simple feature of heavy traffic volume, but may also involve multiple road condition feature knowledge points such as vehicle speed generally below a certain threshold, vehicle queue length exceeding a certain distance, and turning vehicle congestion at a specific intersection; the label of "slow traffic due to bad weather" may cover multiple detailed feature descriptions such as heavy rainfall, low visibility, slippery road surface, and some vehicles slowing down due to cautious driving; the label of "special road conditions during holidays" may include complex situations such as a sharp increase in traffic volume on sections near tourist attractions, vehicle saturation in highway service areas, and an increase in the proportion of out-of-town vehicles.

[0021] For a specific highway road condition learning data, its road condition feature annotation data may include multiple road condition feature labels, such as "local congestion caused by accidents", "adverse effects on nighttime visibility" and "lane change restrictions caused by road maintenance".

[0022] Step S120, encode and represent each of the highway road condition learning data, and generate a coding vector sequence for each of the highway road condition learning data, wherein the coding vector sequence of the highway road condition learning data includes a sample road condition state trend coding vector of the sample multi-dimensional acquisition fusion data, a road condition feature label coding vector of the sample multi-dimensional acquisition fusion data, and a road condition severity coding vector of the sample multi-dimensional acquisition fusion data.

[0023] In detail, the coding vector sequence is the result of encoding the highway road condition learning data, which is expressed in the form of a numerical vector and jointly describes the various characteristics of the road condition, providing a basis for subsequent model learning and prediction.

[0024] The sample road condition state trend coding vector is a coded representation of the road condition state trend information in the sample multi-dimensional acquisition and fusion data, which may include trend information such as traffic volume and speed that changes over time, reflecting the dynamic changes in road conditions. The road condition feature label coding vector is a coded representation of the road condition feature label. Each road condition feature label is converted into a numerical vector to facilitate model learning and prediction. This coding vector contains the semantic information of the label and the association information with other labels. The road condition severity coding vector is a coded representation of the road condition severity. It converts different levels of road condition severity into the form of numerical vectors, which is convenient for the model to perform quantitative analysis and prediction.

[0025] In this embodiment, for traffic flow information, the server may decompose it according to different time periods and road sections, for example, dividing a day into morning peak, noon, evening peak and night time periods, and the traffic flow of different road sections in each time period is used as a characteristic trajectory segment. For speed distribution, it may be divided according to different speed intervals to form multiple characteristic trajectory segments. For weather conditions, sunny days, rainy days, foggy days, etc. may be encoded as different values. For road construction conditions and accident information, they may be decomposed with location coordinates and occurrence time as key features.

[0026] After generating multiple feature trajectory segments, the specific content in the road condition feature annotation data (such as road condition feature labels and road condition severity, etc.) is replaced with shielding placeholders. Then, each feature trajectory segment is used as a reference coding unit, and the shielding placeholders and these reference coding units are associated with the data nodes in the highway road condition learning data to generate a to-be-coded set of the highway road condition learning data, and a starting coding unit is inserted at the beginning of the to-be-coded set.

[0027] Next, the server performs feature mapping on the set to be coded. For example, by using a pre-trained neural network model, each feature trajectory segment is mapped into a numerical vector, thereby generating a data coding vector distribution. At the same time, information such as the location and order of the data nodes is also coded to generate a node coding vector distribution.

[0028] Finally, the data encoding vector distribution and the node encoding vector distribution are fused. For example, the encoding vector sequence of highway road condition learning data can be generated by weighted summation, splicing, etc., including the sample road condition state trend encoding vector (reflecting the changing trend of traffic flow, speed, etc.), the road condition feature label encoding vector (encoding complex road condition feature label information) and the road condition severity encoding vector (indicating the severity level of the road condition).

[0029] Step S130, based on the sample road condition state trend coding vector and the road condition feature label coding vector of the sample multi-dimensional acquisition and fusion data, execute the first network learning task of the highway road condition prediction network, wherein the first network learning task is used to perform parameter learning on the highway road condition prediction network so that the highway road condition prediction network predicts the road condition feature label corresponding to the sample multi-dimensional acquisition and fusion data.

[0030] In this embodiment, the server performs the first network learning task of the highway road condition prediction network based on the sample road condition state trend coding vector and the road condition feature label coding vector of the sample multi-dimensional collection and fusion data.

[0031] Assume that the long short-term memory subnetwork of the highway traffic prediction network receives the sample traffic state trend encoding vector and traffic feature label encoding vector. The long short-term memory subnetwork processes these encoding vectors, extracts the deep features, and generates the first fusion encoding vector. This process is similar to comprehensive analysis and understanding of the input information, capturing the time series features and complex associations.

[0032] After receiving the first fused coding vector, the first mapping sub-network maps it into predicted road condition feature label data. For example, it is predicted that the current road condition may belong to labels such as "severe congestion during peak hours" and "moderate slow traffic caused by bad weather".

[0033] The server determines the first error value between the predicted traffic feature label data and the actual traffic feature label. This may involve detailed comparison and error calculation for each possible traffic feature label. For example, for the label "severe congestion during peak hours", the server will compare the predicted judgment results on traffic flow, speed, queue length, etc. with the actual marked judgment results, calculate the error of each traffic feature knowledge point, and obtain the comprehensive first error value through weighting and other methods.

[0034] Based on the first error value, the server optimizes the network weights and bias information of the long short-term memory subnetwork and the first mapping subnetwork. By continuously adjusting these parameters, the network can more accurately predict the road condition feature label. If the long short-term memory subnetwork and the first mapping subnetwork meet the training termination conditions (for example, the error is less than a certain threshold or reaches a preset number of training rounds), the first network learning task is terminated.

[0035] Step S140, based on the sample road condition state trend coding vector and the road condition severity coding vector of the sample multi-dimensional acquisition and fusion data, run the second network learning task of the highway road condition prediction network, the second network learning task is used to perform parameter learning on the highway road condition prediction network so that the highway road condition prediction network predicts the road condition severity of the sample multi-dimensional acquisition and fusion data, so that the highway road condition prediction network that has completed parameter learning predicts the road condition feature label and road condition severity of any target multi-dimensional acquisition and fusion data.

[0036] In this embodiment, the server runs the second network learning task of the highway road condition prediction network based on the sample road condition state trend coding vector and the road condition severity coding vector of the sample multi-dimensional collection and fusion data.

[0037] The self-attention sub-network in the server first receives the sample road condition state trend encoding vector and the road condition severity encoding vector, and performs feature extraction to generate a second fusion encoding vector. In this process, the self-attention sub-network can automatically focus on the most important part of the input encoding vector and extract key features related to the road condition severity.

[0038] The second mapping subnetwork receives the second blended code vector and maps it into predicted road condition severity data, for example, predicting the severity of the current road condition to be "serious", "moderate" or "mild".

[0039] The server determines a second error value between the predicted traffic severity data and the actual traffic severity. This may require a detailed comparison and calculation of the traffic severity under each traffic feature label. For example, for the label "Congestion during rush hour", the server will compare the predicted severity level with the actual labeled severity level and calculate the error.

[0040] Based on the second error value, the server optimizes the network weights and bias information of the self-attention subnetwork and the second mapping subnetwork. By continuously adjusting these parameters, the network can more accurately predict the severity of the road condition. After multiple iterations and optimizations, when the highway road condition prediction network completes parameter learning, the server can use the network to predict any target multi-dimensional collection and fusion data, and obtain its road condition feature label and road condition severity, providing strong support for traffic management and travel planning.

[0041] Based on the above steps, the embodiment of the present application significantly improves the accuracy and real-time performance of highway road condition monitoring by integrating multi-dimensional road condition data and performing deep learning and fusion processing. Specifically, by introducing multi-dimensional annotation data of road condition feature labels and road condition severity, combined with the fine coding of sample road condition state trends, the complex characteristics of road condition changes can be captured more comprehensively, thereby greatly improving the accuracy of road condition prediction. In addition, the embodiment of the present application is not limited to the monitoring of specific sections or time periods, but the highway road condition prediction network trained by a large amount of learning data has a strong generalization ability and can effectively predict the road conditions of different highways and different time periods. The multi-dimensional collected data (such as traffic flow, speed, weather conditions, etc.) is fused and processed to form a comprehensive road condition status representation, which overcomes the limitation of incomplete information of a single data source and provides a richer and more accurate data basis for road condition monitoring. The entire monitoring process is highly automated, and real-time monitoring and prediction of road conditions can be achieved without human intervention, which significantly improves traffic management efficiency and reduces labor costs. In addition, for complex road conditions, multiple road condition feature labels can be predicted simultaneously, not limited to a single state description, providing traffic management departments with more detailed road condition analysis reports, which helps to formulate more refined traffic management strategies. In other words, through multi-dimensional data fusion and deep learning methods, the comprehensiveness, accuracy and real-time monitoring of highway road conditions are achieved, providing strong technical support for improving traffic management level and public travel experience.

[0042] In a possible implementation, the road condition feature annotation data of the sample multi-dimensional acquisition fusion data also includes the road condition feature knowledge points of the sample multi-dimensional acquisition fusion data under the road condition feature label, and each of the road condition feature labels includes a plurality of the road condition feature knowledge points. The encoding vector sequence of the highway road condition learning data also includes the road condition feature knowledge point encoding vectors of the road condition feature knowledge points.

[0043] Step S140 includes: running the second network learning task of the highway road condition prediction network based on the sample road condition state trend coding vector, the road condition severity coding vector and the road condition feature knowledge point coding vector of the sample multi-dimensional acquisition and fusion data, and the second network learning task is also used to predict the road condition feature knowledge point of the sample multi-dimensional acquisition and fusion data under the road condition feature label when predicting the road condition feature label of the sample multi-dimensional acquisition and fusion data.

[0044] In this embodiment, the server first obtains a wealth of highway road condition learning data. Assume that one of the sample multi-dimensional acquisition and fusion data is about the situation of a certain section of highway in a specific time period (such as 9 am to 10 am). This data contains a traffic volume of 150 vehicles per hour, an average speed of 60 kilometers per hour, light rain, road construction conditions of repairing the right lane, and a minor rear-end collision. Its road condition feature label is "slow driving during peak hours", and the corresponding road condition severity is "moderate".

[0045] Under this road condition feature label, multiple road condition feature knowledge points are included, such as traffic flow approaching saturation, vehicle speed lower than normal, light rain causing slippery road surface affecting driving speed, road construction occupying some lanes resulting in reduced traffic capacity, accidents causing local temporary congestion, etc.

[0046] The server encodes these data. When generating the encoding vector sequence, in addition to the sample road condition state trend encoding vector (an encoding that integrates dynamic information such as traffic flow and speed) and the road condition severity encoding vector (an encoding that indicates moderate), the road condition feature knowledge point encoding vector is also generated.

[0047] The road condition feature knowledge point "traffic flow is close to saturation" may be encoded as a specific numerical vector; "vehicle speed is lower than normal level" also has its corresponding code; similarly, "light rain makes the road surface slippery and affects driving speed", "road construction occupies some lanes, resulting in reduced traffic capacity" and "accidents cause local short-term congestion" and other knowledge points have their own codes.

[0048] Next, the server runs the second network learning task of the highway traffic prediction network.

[0049] The self-attention sub-network receives the sample road condition state trend coding vector, road condition severity coding vector and road condition feature knowledge point coding vector of the sample multi-dimensional collection fusion data. By analyzing these coding vectors, the self-attention sub-network can automatically focus on the key parts, extract important features related to the road condition feature labels and road condition feature knowledge points, and generate the third fusion coding vector.

[0050] The second mapping sub-network performs mapping based on the third fusion coding vector to generate predicted road condition severity data, for example, the predicted result is "moderate".

[0051] The third mapping subnetwork also maps based on the third fusion coding vector to generate prediction data for road condition feature knowledge points. For example, for the knowledge point "traffic flow is close to saturation", the prediction result may be "yes"; for "vehicle speed is lower than normal", the prediction result is also "yes"; for other knowledge points, corresponding predictions are also given.

[0052] The server then determines the error between the prediction and the actual situation.

[0053] For the predicted road condition severity data, the server compares it with the actual road condition severity "moderate". First, the data in the highway road condition learning data sequence is divided into multiple training groups. Assume that it is divided into 10 groups.

[0054] For each sample multi-dimensional acquisition fusion data in each training group, the server calculates the error between the predicted road condition severity and the actual road condition severity. For example, in a certain group, for the current sample multi-dimensional acquisition fusion data, the predicted road condition severity is "mild", which deviates from the actual "moderate". Calculate the square of this error result and then divide it by the number of highway road condition learning data in the training group. Assuming that there are 50 data in this group, then this calculation result is the road condition severity error value of the sample multi-dimensional acquisition fusion data corresponding to this road condition feature label.

[0055] The road condition severity error values ​​of all road condition feature labels are weighted to obtain a second error value between the predicted road condition severity data and the actual road condition severity.

[0056] For the prediction data of road condition feature knowledge points, the server also performs detailed comparison and error calculation. It traverses each road condition feature knowledge point. For example, for the knowledge point "traffic flow is close to saturation", the actual mark is "yes", while the prediction result is "no".

[0057] The server divides the highway road condition learning data sequence into multiple training groups. For the multi-dimensional acquisition fusion data of the sample in each training group, the second exponential conversion value of the prediction result is calculated (for example, by exp(0), assuming that the prediction result is "no" and is encoded as 0), and then the second dot product result of the second exponential conversion value and the actual label ("yes", assuming that it is encoded as 1) is determined (which is 0).

[0058] Divide the second dot product result by the number of data in the training group to obtain the third comparison weight. Assuming that there are 100 data in the training group and the second dot product result is 0, the third comparison weight is 0 / 100=0. The inverse of the third comparison weight is used as the knowledge point discrimination error value between the discrimination result of the road condition feature knowledge point and the actual marked discrimination result.

[0059] The knowledge point discrimination error value of each road condition feature knowledge point is weighted to obtain a third error value.

[0060] The server fuses the first error value (the discrimination error regarding the road condition feature label), the second error value (the error regarding the road condition severity), and the third error value (the error regarding the road condition feature knowledge point) to generate a global error value.

[0061] For example, the global error value can be calculated by weighted summation. Assume that the weight of the first error value is 0.4, the weight of the second error value is 0.3, and the weight of the third error value is 0.3. If the first error value is 0.2, the second error value is 0.15, and the third error value is 0.1, then the global error value is 0.4*0.2+0.3*0.15+0.3*0.1=0.175.

[0062] Based on this global error value, the server optimizes the network weights and bias information of the self-attention sub-network, the second mapping sub-network, and the third mapping sub-network through the back-propagation algorithm.

[0063] By continuously performing such learning tasks and adjusting network parameters, the highway traffic condition prediction network can more accurately predict traffic condition feature labels, traffic condition severity, and traffic condition feature knowledge points.

[0064] After multiple rounds of training and optimization, when the server receives new highway traffic data, it can quickly and accurately provide a comprehensive and detailed traffic analysis, including traffic feature labels (such as "congestion during peak hours"), traffic severity (such as "severe"), and specific traffic feature knowledge points (such as "heavy traffic volume and slow speed", "congestion at multiple intersections", etc.), providing more valuable information and decision-making support for traffic management departments and travelers.

[0065] Suppose in another example, the data obtained by the server is about the highway traffic at night. The traffic volume is light, but the speed is significantly restricted due to the foggy weather. The road condition feature label is "low traffic efficiency in bad weather" and the road condition severity is "mild".

[0066] The road condition feature knowledge points under this road condition feature label include "heavy fog causes low visibility" and "vehicles drive cautiously and actively slow down".

[0067] Similarly, the server encodes this data and inputs it into the highway traffic prediction network for learning and prediction. In the process of calculating errors and optimizing network parameters, similar methods as above are used to continuously improve the network's prediction ability to adapt to various complex highway traffic scenarios.

[0068] Another example is about highway traffic during holidays. The traffic volume increases significantly, and some sections of the road are blocked due to traffic accidents. The traffic feature label is "Holiday peak congestion" and the traffic severity is "severe".

[0069] Knowledge points about its road condition characteristics include "concentration of tourist vehicles" and "long accident handling time".

[0070] The server processes and learns this data according to established procedures, and continuously improves and optimizes the highway traffic condition prediction network, so that it can respond to various types of highway traffic conditions more accurately and provide better services for traffic management and public travel.

[0071] Therefore, by comprehensively considering the sample road condition state trend coding vector, road condition severity coding vector and road condition feature knowledge point coding vector of the sample multi-dimensional collection and fusion data, the second network learning task of the highway road condition prediction network is run, the network parameters are continuously optimized, and the accuracy and comprehensiveness of the road condition prediction are improved.

[0072] In a possible implementation, the highway traffic condition prediction network includes: a long short-term memory subnetwork, a first mapping subnetwork, a self-attention subnetwork, and a second mapping subnetwork.

[0073] Step S130 includes:

[0074] Step A110, running the long short-term memory sub-network to extract features based on the sample road condition state trend coding vector and the road condition feature label coding vector of the sample multi-dimensional acquisition fusion data to generate a first fusion coding vector.

[0075] Step A120: Run the first mapping subnetwork to perform mapping according to the first fusion coding vector to generate predicted road condition feature label data.

[0076] Step A130, determine the first error value between the predicted road condition feature label data and the road condition feature label of the sample multi-dimensional acquisition fusion data, and optimize the network weights and bias information of the long short-term memory subnetwork and the network weights and bias information of the first mapping subnetwork based on the first error value.

[0077] Step A140: If the long short-term memory sub-network meets the training termination condition and the first mapping sub-network meets the training termination condition, the first network learning task is terminated.

[0078] In this embodiment, when performing the highway traffic condition prediction task, the server first constructs a highway traffic condition prediction network including a long short-term memory subnetwork, a first mapping subnetwork, a self-attention subnetwork, and a second mapping subnetwork. The highway traffic condition prediction network is designed to comprehensively and accurately predict highway traffic condition feature labels and traffic condition severity, providing data support for traffic management and public travel.

[0079] The server collects a large amount of First Expressway traffic learning data from various sensors distributed along the expressway and the database of the traffic management department. These First Expressway traffic learning data include multi-dimensional collection and fusion information, such as traffic flow, speed distribution, weather conditions, road construction conditions, and the location and type of accidents.

[0080] Taking a section of highway in a specific time period (such as 8:00 to 9:00 a.m.) as an example, the server obtains the following data: the traffic volume is 200 vehicles per hour, the average speed is 80 kilometers per hour, the weather is sunny, there is no road construction, and the accident is a minor scratch between two vehicles. The corresponding road condition feature label is "smooth traffic during the morning rush hour" and the road condition severity is "mild".

[0081] The server encodes these data and generates sample road condition status trend encoding vectors and road condition feature label encoding vectors of sample multi-dimensional acquisition fusion data. The sample road condition status trend encoding vector integrates the encoding of dynamic information such as traffic volume and speed, while the road condition feature label encoding vector encodes the label of "smooth traffic during the morning rush hour".

[0082] Next, the server inputs the two encoding vectors into the LSTM sub-network, which is a special type of recurrent neural network that can process sequence data and capture the time series features and complex associations therein.

[0083] In the LSTM subnetwork, the sample road condition state trend encoding vector and the road condition feature label encoding vector are processed step by step. Each time step in the network receives a portion of the input data and updates the current hidden state based on the current input and the previous hidden state. Through this processing, the LSTM subnetwork can extract the deep features in the input encoding vector and generate the first fusion encoding vector.

[0084] This first fusion coding vector is a coded representation that combines the sample road condition state trend and road condition feature label information. It captures the key features in the input data and the correlation between them, providing an important feature basis for subsequent road condition feature label prediction.

[0085] Once the long short-term memory sub-network generates the first fusion encoding vector, the server will input this vector into the first mapping sub-network. The first mapping sub-network is a fully connected neural network whose task is to map the first fusion encoding vector into predicted road condition feature label data.

[0086] In the first mapping subnetwork, the first fusion coding vector is processed by multiple fully connected layers, each of which performs linear transformation and nonlinear activation operations. Through such processing, the first mapping subnetwork can extract key features in the first fusion coding vector and map it to predicted road condition feature label data.

[0087] Taking the previous data as an example, the first mapping subnetwork may output a prediction result representing the traffic feature label of "smooth traffic during the morning rush hour". This prediction result is a numerical vector, in which each element corresponds to a possible traffic feature label, and the value of the element represents the probability or confidence of the prediction of the label.

[0088] After the server generates the predicted road condition feature label data, it will compare it with the actual road condition feature label of the sample multi-dimensional acquisition fusion data to determine the first error value. This first error value is a measure of the deviation between the predicted result and the actual label, which reflects the accuracy of the network in predicting the road condition feature label.

[0089] To calculate the first error value, the server may use the cross entropy loss function or other suitable loss functions. The cross entropy loss function measures the difference between the predicted probability distribution and the actual label distribution, and is particularly effective for multi-classification problems.

[0090] Once the first error value is calculated, the server will optimize the network weights and bias information of the long short-term memory subnetwork and the first mapping subnetwork based on this error value. This is achieved through the back-propagation algorithm, which can calculate the gradient of the error with respect to the network weights and update the network weights and bias information through gradient descent or other optimization algorithms.

[0091] Through this optimization process, the long short-term memory subnetwork and the first mapping subnetwork can gradually learn how to more accurately predict the road condition feature labels. As the training progresses, the network weights and bias information will be continuously adjusted, making the prediction results closer and closer to the actual labels.

[0092] During the training process, the server continuously monitors the performance of the LSTM subnetwork and the first mapping subnetwork. This usually involves monitoring indicators such as prediction accuracy and loss value on the validation set or test set.

[0093] Once the long short-term memory sub-network and the first mapping sub-network meet the predetermined training termination conditions (such as the accuracy on the validation set no longer improves, the preset number of training rounds is reached, etc.), the server will end the first network learning task. At this point, the long short-term memory sub-network and the first mapping sub-network have learned how to predict the road condition feature label based on the road condition state trend encoding vector and road condition feature label encoding vector of the sample multi-dimensional collection fusion data.

[0094] Through the first network learning task, the server has laid a solid foundation for the highway traffic prediction network, enabling it to accurately predict various traffic feature labels. This provides strong support for subsequent traffic severity prediction and traffic feature knowledge point prediction, and brings greater convenience and value to traffic management and public travel.

[0095] In a possible implementation, step S140 includes:

[0096] Step S141, running the self-attention sub-network to extract features based on the sample road condition state trend coding vector and the road condition severity coding vector of the sample multi-dimensional collected fusion data to generate a second fusion coding vector.

[0097] Step S142: Run the second mapping subnetwork to perform mapping according to the second fusion coding vector to generate predicted road condition severity data.

[0098] Step S143, determine a second error value between the predicted road condition severity data and the road condition severity of the sample multi-dimensional acquisition fusion data, and optimize the network weights and bias information of the self-attention subnetwork and the network weights and bias information of the second mapping subnetwork based on the second error value.

[0099] In this embodiment, the server first extracts the road condition state trend coding vector and the road condition severity coding vector from the prepared sample multi-dimensional collection fusion data. These data are collected by various sensors and monitoring equipment, such as traffic flow, vehicle speed, weather conditions, etc., and have been pre-processed and encoded and converted into a format suitable for neural network processing.

[0100] The server then feeds these encoded vectors into the self-attention subnetwork, a special neural network structure that can extract features from the input data and capture complex patterns and associations in the data. In the process of extracting features, the self-attention subnetwork pays attention to different parts of the input data and weights them according to their importance.

[0101] After processing by the self-attention sub-network, the server generates a new encoding vector, the second fusion encoding vector. This second fusion encoding vector combines the characteristics of the road condition trend and the road condition severity, providing rich information for subsequent road condition prediction.

[0102] After obtaining the second blended code vector, the server will input it into the second mapping sub-network. The task of the second mapping sub-network is to map the blended code vector to the predicted value of the road condition severity. This process is similar to a function mapping, which converts the input code vector into a specific value, indicating the predicted road condition severity.

[0103] The second mapping subnetwork may include multiple fully connected layers or convolutional layers. Through the processing of these layers, the server can gradually extract the key information in the fusion coding vector and convert it into predicted road condition severity data.

[0104] After generating the predicted road condition severity data, the server will compare it with the actual road condition severity in the sample multi-dimensional acquisition fusion data, and calculate the error value between the two, that is, the second error value. This second error value reflects the accuracy of the prediction result and is an important basis for optimizing the neural network. In order to reduce the error value and improve the accuracy of the prediction, the server will optimize the network weights and bias information of the self-attention subnetwork and the second mapping subnetwork according to the second error value. This process is usually implemented using the back propagation algorithm and the gradient descent algorithm to minimize the error value by continuously adjusting the network parameters. After multiple iterations and optimizations, the server's prediction ability will be improved, and it will be able to more accurately predict the severity of highway road conditions.

[0105] In a possible implementation, the predicted road condition feature label data includes the judgment results of the sample multi-dimensional acquisition fusion data corresponding to multiple road condition feature labels.

[0106] Step A130 includes:

[0107] Step A131, traverse each of the multiple road condition feature labels, and determine the discrimination error value between the discrimination result of the road condition feature label of the sample multi-dimensional acquisition fusion data and the real labeled discrimination result of the road condition feature label.

[0108] Step A132, determining a weighted result of the discrimination error value of each of the road condition feature labels to serve as a first error value between the predicted road condition feature label data and the road condition feature label of the sample multi-dimensional acquisition fusion data.

[0109] In this embodiment, when the server performs the road condition prediction task, it will generate a series of predicted road condition feature label data. These predicted road condition feature label data are obtained based on the sample multi-dimensional collection and fusion data, and each sample multi-dimensional collection and fusion data corresponds to the discrimination results of multiple road condition feature labels. These road condition feature labels may include a variety of road condition features such as slippery road surface, traffic congestion, construction occupying the road, etc., and the discrimination result is the judgment whether these features exist.

[0110] The server checks each traffic feature label one by one to ensure that the prediction results of each traffic feature label are evaluated.

[0111] For each road condition feature label, the server will compare the predicted discrimination result with the real labeled discrimination result. The real labeled discrimination result is pre-labeled based on the actual road condition data and is used as a benchmark for evaluating the predicted results. By comparing the two, the server can calculate the discrimination error value, which reflects the accuracy of the prediction result.

[0112] Since different traffic feature labels may have different importance for traffic prediction, the server will assign a weight to each discrimination error value. This weight reflects the importance of the traffic feature label in the prediction task. Through weighted processing, the server can obtain a comprehensive error value, namely the first error value. This first error value comprehensively considers the prediction accuracy of all traffic feature labels and is an important indicator for evaluating the performance of the entire prediction model.

[0113] Therefore, by calculating the first error value between the predicted road condition feature label data and the actual situation, the accuracy of the prediction model can be comprehensively evaluated, and an important basis can be provided for subsequent model optimization.

[0114] In a possible implementation, step A131 includes:

[0115] Step A1311, dividing the highway road condition learning data in the highway road condition learning data sequence into multiple training groups.

[0116] Step A1312, traverse the sample multi-dimensional acquisition fusion data in each of the highway road condition learning data in each of the training groups, determine the first exponential conversion value of the discrimination result of the road condition feature label of the sample multi-dimensional acquisition fusion data, and determine the first dot product result of the first exponential conversion value and the true label discrimination result of the sample multi-dimensional acquisition fusion data.

[0117] Step A1313, determine the first comparison weight between the first dot product result and the scale of the highway road condition learning data included in the training group, and use the first comparison weight as the discrimination error value between the discrimination result of the road condition feature label of the sample multi-dimensional acquisition fusion data and the true annotation discrimination result.

[0118] In this embodiment, when processing the highway traffic condition learning data, the server first groups the highway traffic condition learning data sequences for processing. This is to conduct model training more efficiently. By dividing the highway traffic condition learning data into multiple groups, the server can process the highway traffic condition learning data in parallel, thereby increasing the training speed. Each training group contains a certain amount of highway traffic condition learning data, which are usually divided according to time sequence or a certain specific rule when grouping.

[0119] Next, the server will check the data in each training group one by one. For each sample multi-dimensional acquisition fusion data in the highway road condition learning data, the server will perform a series of processing to determine the discrimination results of the corresponding road condition feature labels. In this process, the server may use neural networks or other machine learning models to extract features and make discriminations. After obtaining the discrimination results, the server will perform exponential conversion on these results to obtain the first exponential conversion value. This conversion is to map the data to a numerical range that is more suitable for subsequent calculations.

[0120] After obtaining the first exponential conversion value, the server will perform a dot product operation on it and the real annotation discrimination result of the sample multi-dimensional acquisition fusion data. The real annotation discrimination result is annotated in advance based on the actual road condition data and is used as a benchmark for evaluating the prediction results. The dot product operation is a mathematical operation that multiplies the corresponding elements of two vectors and then sums them to obtain a scalar result. In this scenario, the dot product result reflects the similarity between the predicted discrimination result and the real annotation discrimination result.

[0121] After calculating the first dot product result, the server will also consider the size of the training group. The size of the training group refers to the number of highway road condition learning data contained in the group. The server will adjust the weight of the first dot product result according to the size of the training group to obtain the first comparison weight. This weight reflects the relative error between the predicted discrimination result and the true labeled discrimination result under the current training group size.

[0122] Finally, the server uses the first comparison weight as the discrimination error value. This discrimination error value reflects the degree of error between the predicted discrimination result of the road condition feature label of the sample multi-dimensional collection fusion data and the actual labeled discrimination result under the current training group. By calculating this error value, the server can evaluate the accuracy of the prediction model and provide an important basis for subsequent model optimization.

[0123] In a possible implementation, the highway traffic condition prediction network further includes a self-attention subnetwork, a second mapping subnetwork, and a third mapping subnetwork.

[0124] The second network learning task of the highway road condition prediction network is run based on the sample road condition state trend coding vector, the road condition severity coding vector and the road condition feature knowledge point coding vector of the sample multi-dimensional acquisition fusion data, including:

[0125] Step B110, running the self-attention sub-network to extract features based on the sample road condition state trend coding vector, the road condition severity coding vector and the road condition feature knowledge point coding vector of the sample multi-dimensional collected fusion data to generate a third fusion coding vector.

[0126] Step B120: Run the second mapping subnetwork to perform mapping according to the third fusion coding vector to generate predicted road condition severity data.

[0127] Step B130, running the third mapping sub-network to perform mapping according to the third fusion coding vector to generate road condition feature knowledge point prediction data.

[0128] Step B140, determining a second error value between the predicted road condition severity data and the road condition severity of the sample multi-dimensional acquisition fusion data, and determining a third error value between the road condition feature knowledge point prediction data and the road condition feature knowledge point of the sample multi-dimensional acquisition fusion data.

[0129] Step B150, performing a fusion calculation on the first error value, the second error value and the third error value to generate a global error value.

[0130] Step B160, optimizing the network weights and bias information of the self-attention sub-network, the network weights and bias information of the second mapping sub-network, and the network weights and bias information of the third mapping sub-network according to the global error value.

[0131] In this embodiment, the highway traffic prediction network is composed of multiple sub-networks, including a self-attention sub-network, a second mapping sub-network, and a third mapping sub-network. These sub-networks each undertake different tasks and work together to complete complex traffic prediction.

[0132] After receiving the sample multi-dimensional acquisition fusion data, the server extracts the road condition state trend encoding vector, road condition severity encoding vector, and road condition feature knowledge point encoding vector. These encoding vectors are generated in the data preprocessing stage and represent different aspects of the road condition. The server inputs these encoding vectors into the self-attention sub-network, which extracts features from these encoding vectors to capture the complex relationships and patterns between them. After feature extraction, the self-attention sub-network generates a new encoding vector, namely the third fusion encoding vector, which integrates the features of road condition state trend, road condition severity, and road condition feature knowledge points.

[0133] After obtaining the third blended code vector, the server will input it into the second mapping sub-network. The task of the second mapping sub-network is to map the blended code vector to the predicted value of the road condition severity. Through a series of calculations and conversions, the second mapping sub-network can generate predicted road condition severity data, which represents the server's prediction results of the road condition severity.

[0134] At the same time, the server will also input the third blended coding vector into the third mapping sub-network. The task of the third mapping sub-network is to map the blended coding vector to the predicted value of the road condition feature knowledge point. Through processing, the third mapping sub-network can generate road condition feature knowledge point prediction data, which represents the server's prediction results for the road condition feature knowledge point.

[0135] After generating the predicted road condition severity data and the road condition feature knowledge point prediction data, the server will compare them with the actual road condition severity and road condition feature knowledge points in the sample multi-dimensional acquisition fusion data. Through calculation, the server can determine the second error value between the predicted road condition severity data and the actual road condition severity, and the third error value between the road condition feature knowledge point prediction data and the actual road condition feature knowledge points. These two error values ​​reflect the accuracy of the prediction results.

[0136] After obtaining the first error value (the discrimination error value of the road condition feature label), the second error value (the prediction error value of the road condition severity), and the third error value (the prediction error value of the road condition feature knowledge point), the server will perform a fusion calculation on these three error values. The fusion calculation method may be weighted average, summation, or other customized fusion methods. Through fusion calculation, the server can generate a global error value that comprehensively considers the prediction accuracy of the road condition feature label, road condition severity, and road condition feature knowledge points.

[0137] Finally, the server will optimize the network weights and bias information of the self-attention sub-network, the second mapping sub-network, and the third mapping sub-network according to the global error value. The purpose of the optimization is to reduce the global error value and improve the accuracy of the prediction. The server may use the back propagation algorithm and the gradient descent algorithm for optimization. After multiple iterations and optimizations, the server's prediction ability will be improved, and it will be able to more accurately predict the road conditions of the highway.

[0138] In one possible implementation, the predicted road condition severity data includes the predicted road condition severity of the sample multi-dimensional acquisition fusion data corresponding to multiple road condition feature labels, and the road condition severity of the sample multi-dimensional acquisition fusion data includes the actual annotated road condition severity of the sample multi-dimensional acquisition fusion data corresponding to multiple road condition feature labels.

[0139] Step B140 includes:

[0140] Step B141, traverse each of the plurality of road condition feature labels, and divide the highway road condition learning data in the highway road condition learning data sequence into a plurality of training groups.

[0141] Step B142, traverse the sample multi-dimensional acquisition fusion data in each of the highway road condition learning data in each of the training groups, and determine the error result between the predicted road condition severity data corresponding to the road condition feature label of the sample multi-dimensional acquisition fusion data and the actual labeled road condition severity corresponding to the road condition feature label.

[0142] Step B143, determine the second comparison weight between the square of the error result and the scale of the highway road condition learning data included in the training group, and use the second comparison weight as the road condition severity error value between the predicted road condition severity data corresponding to the road condition feature label of the sample multi-dimensional acquisition fusion data and the actual labeled road condition severity.

[0143] Step B144, determining a weighted result of the road condition severity error value of each road condition feature label as a second error value between the predicted road condition severity data and the road condition severity of the sample multi-dimensional acquisition fusion data.

[0144] In this embodiment, when processing the highway road condition prediction task, predicted road condition severity data will be generated. These predicted road condition severity data not only include the prediction of the severity of the entire road condition, but also refine the predicted road condition severity corresponding to each road condition feature label. At the same time, the sample multi-dimensional acquisition fusion data also includes the real annotated road condition severity corresponding to multiple road condition feature labels, which is used as a benchmark for evaluating the accuracy of the prediction.

[0145] The server will start to traverse multiple defined traffic feature labels. These labels represent different aspects of traffic conditions, such as congestion level, road surface conditions, etc. For each label, the server will perform subsequent calculation steps to evaluate the accuracy of the prediction.

[0146] For each traffic feature label, the server divides the highway traffic learning data sequence into multiple training groups. This is to train the model more efficiently. By grouping the data, the server can process the data in parallel, thereby improving computing efficiency.

[0147] Next, the server will traverse the highway traffic learning data in each training group. For each data point, that is, the sample multi-dimensional collection and fusion data, the server will extract the predicted traffic severity and the actual annotated traffic severity corresponding to the current traffic feature label.

[0148] The server will calculate the error between the predicted road condition severity and the actual annotated road condition severity. This error reflects the difference between the predicted value and the actual value.

[0149] To more comprehensively evaluate the prediction accuracy, the server considers not only the error result itself but also the size of the training group. The server determines a second comparison weight between the square of the error result and the size of the highway traffic learning data in the training group. This weight reflects both the size of the prediction error and the impact of the data size.

[0150] The server uses the second comparison weight as the road condition severity error value between the predicted road condition severity data corresponding to the current road condition feature label of the sample multi-dimensional acquisition fusion data and the actual annotated road condition severity. This error value more comprehensively reflects the gap between the predicted value and the actual value under the current training group.

[0151] Finally, the server will traverse all the road condition feature tags and calculate the weighted result of the road condition severity error value corresponding to each tag. This weighted result comprehensively considers the prediction accuracy of all road condition feature tags and serves as the second error value between the predicted road condition severity data and the road condition severity of the sample multi-dimensional acquisition fusion data. This second error value is an important indicator for evaluating the accuracy of the entire prediction model.

[0152] In a possible implementation manner, the road condition characteristic knowledge point prediction data includes the judgment results of the sample multi-dimensional acquisition and fusion data corresponding to multiple road condition characteristic knowledge points.

[0153] Step B140 further includes:

[0154] Step B145, traverse each road condition feature knowledge point in the multiple road condition feature knowledge points, and determine the knowledge point discrimination error value between the discrimination result of the road condition feature knowledge point and the real labeled discrimination result of the road condition feature knowledge point of the sample multi-dimensional acquisition fusion data.

[0155] Step B146, determining a weighted result of the knowledge point discrimination error value of each of the road condition feature knowledge points as a third error value between the road condition feature knowledge point prediction data and the road condition feature knowledge point of the sample multi-dimensional acquisition fusion data.

[0156] In this embodiment, when the server processes the highway road condition prediction task, it will not only predict the severity of the road condition, but also distinguish the road condition feature knowledge points. The road condition feature knowledge point prediction data includes the discrimination results of the sample multi-dimensional acquisition fusion data corresponding to multiple road condition feature knowledge points. These discrimination results represent the server's understanding and judgment of the road condition feature knowledge points.

[0157] The server will start to traverse the defined multiple road condition feature knowledge points. These knowledge points may include the slipperiness of the road surface, the recognition of traffic signs, road construction information, etc. For each knowledge point, the server will perform subsequent calculation steps to evaluate the accuracy of the judgment.

[0158] For each road condition feature knowledge point, the server extracts the discrimination result in the road condition feature knowledge point prediction data and compares it with the real labeled discrimination result of the road condition feature knowledge point in the sample multi-dimensional acquisition fusion data. Through calculation, the server can determine the knowledge point discrimination error value between the discrimination result and the real labeled discrimination result. This error value reflects the gap between the discrimination result and the actual situation.

[0159] After traversing all the road condition feature knowledge points and calculating the knowledge point discrimination error value of each knowledge point, the server will weight these error values. The purpose of weighting is to balance the importance of different knowledge points in the overall evaluation. The server will perform weighted summation of the discrimination error value of each knowledge point according to the preset weight distribution, so as to obtain the third error value between the road condition feature knowledge point prediction data and the road condition feature knowledge point of the sample multi-dimensional acquisition fusion data. This third error value is an important indicator for evaluating the accuracy of road condition feature knowledge point discrimination.

[0160] In a possible implementation, step B145 includes:

[0161] Step B1451, dividing the highway road condition learning data in the highway road condition learning data sequence into multiple training groups.

[0162] Step B1452, traverse the sample multi-dimensional acquisition fusion data in each of the highway road condition learning data in each of the training groups, determine the second exponential conversion value of the discrimination result of the road condition feature knowledge point, and determine the second dot product result of the second exponential conversion value and the real annotation discrimination result of the road condition feature knowledge point.

[0163] Step B1453, determining a third comparison weight between the second dot product result and the size of the highway traffic condition learning data included in the training group.

[0164] Step B1454, taking the inverse of the third comparison weight as the knowledge point discrimination error value between the discrimination result of the road condition feature knowledge point and the actual labeled discrimination result of the road condition feature knowledge point of the sample multi-dimensional acquisition fusion data.

[0165] In this embodiment, when processing the highway traffic condition prediction task, the server first divides the highway traffic condition learning data sequence into multiple training groups. This is to more effectively train the model. Through group processing, the server can process these data in parallel, thereby improving computing efficiency. Each training group contains a portion of the highway traffic condition learning data, which is used to train the model and evaluate its performance.

[0166] Next, the server will traverse the highway traffic learning data in each training group. For each data point, that is, the sample multi-dimensional collection fusion data, the server will extract the discrimination result of the corresponding road condition feature knowledge point and the true annotation discrimination result. This is to compare the difference between the discrimination result and the true annotation, so as to evaluate the performance of the model.

[0167] For each road condition feature knowledge point, the server calculates the second exponential conversion value of its judgment result. This is achieved by applying an exponential function to the judgment result, with the purpose of converting the judgment result to a numerical range that is more suitable for subsequent calculations. This conversion value will be used in subsequent dot product calculations.

[0168] The server calculates the second dot product result of the second exponential conversion value and the real annotation discrimination result of the road condition feature knowledge point. The dot product is a mathematical operation used to measure the similarity between two vectors. Here, it is used to compare the similarity between the conversion value of the discrimination result and the real annotation, thereby obtaining a value reflecting the discrimination accuracy.

[0169] To more comprehensively evaluate the accuracy of the discrimination, the server considers not only the dot product result itself but also the size of the training group. The server determines a third comparison weight between the second dot product result and the size of the highway traffic learning data in the training group. This weight reflects both the similarity between the discrimination result and the true annotation and the impact of the data size.

[0170] Finally, the server uses the opposite number of the third comparison weight as the knowledge point discrimination error value between the discrimination result of the road condition feature knowledge point and the real annotation discrimination result of the road condition feature knowledge point of the sample multi-dimensional acquisition fusion data. This error value is a negative number, and its absolute value reflects the difference between the discrimination result and the real annotation. By calculating the knowledge point discrimination error values ​​of all road condition feature knowledge points, the server can evaluate the performance of the model in discriminating road condition feature knowledge points.

[0171] In a possible implementation, step S120 includes:

[0172] Step S121 , traversing each of the highway road condition learning data, performing feature trajectory decomposition on the highway road condition learning data, and generating a plurality of feature trajectory segments.

[0173] Step S122: replacing the traffic feature annotation data in the highway traffic learning data with shielding placeholder symbols.

[0174] Step S123, taking each of the multiple feature trajectory segments as a reference coding unit, the shielding placeholder symbol and the reference coding units corresponding to the multiple feature trajectory segments are respectively associated according to the data nodes in the highway road condition learning data, to generate a to-be-encoded set of the highway road condition learning data, a starting coding unit being inserted at the starting end of the to-be-encoded set.

[0175] Step S124, performing feature mapping processing on the set to be encoded to generate data encoding vector distribution and node encoding vector distribution corresponding to the highway road condition learning data.

[0176] Step S125, generating a coding vector sequence of the highway road condition learning data by fusing the data coding vector distribution and the node coding vector distribution.

[0177] In this embodiment, the server starts to process the highway road condition learning data, and first traverses each highway road condition learning data. During the traversal process, the server performs feature trajectory decomposition on each highway road condition learning data, and splits it into multiple feature trajectory segments. These feature trajectory segments represent different aspects or features of the road condition data, such as traffic flow, vehicle speed, etc.

[0178] After decomposing the feature trajectory segments, the server then replaces the road condition feature annotation data in the highway road condition learning data with shielding placeholders. This step is to make the model focus more on the feature trajectory segments themselves during the subsequent encoding process, rather than relying directly on the annotation data.

[0179] The server uses each of the multiple feature trajectory segments as a reference coding unit. Then, it will shield the placeholder symbol and the reference coding units corresponding to the multiple feature trajectory segments, and associate them according to the data nodes in the highway road condition learning data. In this way, the server generates a set of highway road condition learning data to be encoded. In order to mark the beginning of the encoding, the server also inserts a starting coding unit at the beginning of the set to be encoded.

[0180] After having the set to be encoded, the server then performs feature mapping on it. The purpose of this step is to convert each element in the set to be encoded (including the reference encoding units corresponding to the shielding placeholders and feature trajectory segments) into a data encoding vector distribution and a node encoding vector distribution. These two distributions represent the feature representation of the element at the data level and the node level respectively.

[0181] Finally, the server performs fusion based on the data encoding vector distribution and the node encoding vector distribution. By fusing these two distributions, the server generates an encoding vector sequence of the highway traffic learning data. This encoding vector sequence is a compact and information-rich representation that captures the main features and structures of the highway traffic learning data, facilitating subsequent processing and analysis.

[0182] In a possible implementation, the step S110 includes:

[0183] Step S111, obtaining a previously collected second highway road condition learning data sequence, wherein the second highway road condition learning data sequence includes the sample multi-dimensional acquisition fusion data and the road condition feature labels and road condition severity of the sample multi-dimensional acquisition fusion data.

[0184] Step S112, obtaining the road condition feature knowledge points under the road condition feature labels of the sample multi-dimensional acquisition fusion data.

[0185] Step S113, constructing the road condition feature annotation data of the sample multi-dimensional acquisition and fusion data according to the road condition feature label of the sample multi-dimensional acquisition and fusion data, the road condition severity of the sample multi-dimensional acquisition and fusion data, and the road condition feature knowledge points of the sample multi-dimensional acquisition and fusion data under the road condition feature label.

[0186] Step S114, constructing highway road condition learning data based on the sample multi-dimensional collected fusion data and the road condition feature annotation data, and constructing the first highway road condition learning data sequence based on the multiple highway road condition learning data.

[0187] In this embodiment, the server first obtains the second highway road condition learning data sequence from the previously collected data. This second highway road condition learning data sequence contains rich information, including sample multi-dimensional acquisition and fusion data, road condition feature labels corresponding to these sample multi-dimensional acquisition and fusion data, and labels describing the severity of the road condition. These sample multi-dimensional acquisition and fusion data are the basis for the server to perform subsequent processing and learning.

[0188] After acquiring the road condition learning data sequence of the second highway, the server further extracts the road condition feature knowledge points under the corresponding road condition feature labels of the sample multi-dimensional acquisition fusion data. These road condition feature knowledge points are more detailed descriptions of road condition features, which provide the server with more details and in-depth understanding of the road conditions.

[0189] The server uses the road condition feature labels, road condition severity, and road condition feature knowledge points of the sample multi-dimensional collection fusion data to form the road condition feature annotation data of the sample multi-dimensional collection fusion data. This step is a key link in data preprocessing, which provides accurate and comprehensive annotation information for subsequent learning tasks.

[0190] After obtaining the road condition feature annotation data, the server combines it with the sample multi-dimensional collection and fusion data to form complete highway road condition learning data. These data contain both the original collection and fusion data and detailed annotation information, which is an ideal data set for the server to learn and train.

[0191] Finally, the server organizes multiple highway road condition learning data in a certain order and rules to form the first highway road condition learning data sequence. This data sequence is the basis for the server to perform tasks such as deep learning and pattern recognition, and it will provide strong data support for the prediction and judgment of highway road conditions.

[0192] In a possible implementation, step S112 includes:

[0193] Step S1121, obtaining a previously configured road condition information comparison library, the road condition information comparison library includes association information between different road condition feature labels, different road condition severity levels and different road condition feature knowledge points. Search the association information based on the road condition feature label and the road condition severity level of the sample multi-dimensional acquisition fusion data, and generate the road condition feature knowledge point of the sample multi-dimensional acquisition fusion data under the road condition feature label. Or,

[0194] Step S1122, using a pre-trained scene recognition model, the road condition feature knowledge points of the sample multi-dimensional collected fusion data are identified from a plurality of previously defined road condition feature knowledge points.

[0195] In this embodiment, the server first obtains the traffic information comparison library from a preset storage location or database. The traffic information comparison library is a carefully constructed resource that records in detail the association information between different traffic feature labels, different traffic severity levels and different traffic feature knowledge points.

[0196] The server uses the road condition feature labels and road condition severity in the sample multi-dimensional acquisition fusion data as query conditions and searches in the road condition information reference library. This step is to find road condition feature knowledge points that match specific road condition feature labels and severity. Based on the search results, the server generates road condition feature knowledge points under specific road condition feature labels in the sample multi-dimensional acquisition fusion data. These knowledge points further refine and describe the road condition features, providing valuable information for subsequent data processing and analysis.

[0197] Alternatively, the server loads a pre-trained scene recognition model from the model library. This model is trained based on a large amount of historical data and advanced machine learning algorithms, and it can identify road condition feature knowledge points from multi-dimensional acquisition and fusion data. The server ensures that multiple road condition feature knowledge points have been defined, which cover various possible characteristics of road conditions, such as slippery road surface, traffic congestion, construction areas, etc. The server inputs the sample multi-dimensional acquisition and fusion data into the scene recognition model, and the model starts to process the data and identify road condition feature knowledge points. This step is to use the powerful recognition ability of the model to extract valuable information from the data. After the model recognition is completed, the server obtains the recognition result, that is, the road condition feature knowledge points of the sample multi-dimensional acquisition and fusion data. These knowledge points will be used in subsequent data processing, analysis or learning tasks.

[0198] Figure 2 The hardware structure of the cloud computing system 100 provided in the embodiment of the present application for implementing the above-mentioned highway road condition monitoring method based on multi-dimensional data fusion is shown. Figure 2 As shown, the cloud computing system 100 may include a processor 110 , a machine-readable storage medium 120 , a bus 130 , and a communication unit 140 .

[0199] In one possible design, the cloud computing system 100 can be a single server or a server group. The server group can be centralized or distributed (for example, the cloud computing system 100 can be a distributed system). In some embodiments, the cloud computing system 100 can be local or remote. For example, the cloud computing system 100 can access information and / or data stored in the machine-readable storage medium 120 via a network. For another example, the cloud computing system 100 can be directly connected to the machine-readable storage medium 120 to access the stored information and / or data. In some embodiments, the cloud computing system 100 can be implemented on a cloud computing system. By way of example only, the cloud computing system can include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multi-layer cloud, etc. or any aggregation thereof.

[0200] The machine-readable storage medium 120 may store data and / or instructions. In some embodiments, the machine-readable storage medium 120 may store data obtained from an external terminal. In some embodiments, the machine-readable storage medium 120 may store data and / or instructions used by the cloud computing system 100 to execute or use to complete the exemplary methods described in this application.

[0201] During the specific implementation process, one or more processors 110 execute computer executable instructions stored in the machine-readable storage medium 120, so that the processor 110 can execute the highway road condition monitoring method based on multi-dimensional data fusion in the above method embodiment. The processor 110, the machine-readable storage medium 120 and the communication unit 140 are connected through the bus 130, and the processor 110 can be used to control the sending and receiving actions of the communication unit 140.

[0202] The specific implementation process of the processor 110 can refer to the various method embodiments executed by the above-mentioned cloud computing system 100. The implementation principles and technical effects are similar and will not be repeated in this embodiment.

[0203] In addition, an embodiment of the present application also provides a readable storage medium, in which computer executable instructions are set. When a processor runs the computer executable instructions, the above-mentioned highway road condition monitoring method based on multi-dimensional data fusion is implemented.

[0204] It should be noted that in order to simplify the description disclosed in this application and thus help understand one or more embodiments of the invention, in the foregoing description of the embodiments of the present application, multiple features are sometimes combined into one embodiment, drawings, or descriptions thereof. Similarly, it should be noted that in order to simplify the description disclosed in this application and thus help understand one or more embodiments of the invention, in the foregoing description of the embodiments of the present application, multiple features are sometimes combined into one embodiment, drawings, or descriptions thereof.

Claims

1. A highway road condition monitoring method based on multi-dimensional data fusion, characterized in that: The method comprises: Acquire a first highway road condition learning data sequence, wherein the highway road condition learning data in the first highway road condition learning data sequence includes sample multi-dimensional acquisition fusion data and road condition feature annotation data of the sample multi-dimensional acquisition fusion data, wherein the road condition feature annotation data includes a road condition feature label and a road condition severity of the sample multi-dimensional acquisition fusion data; the road condition feature annotation data of at least one of the highway road condition learning data in the first highway road condition learning data sequence includes a plurality of road condition feature labels; Encode and represent each of the highway road condition learning data to generate a coding vector sequence for each of the highway road condition learning data, wherein the coding vector sequence for the highway road condition learning data includes a sample road condition state trend coding vector of the sample multi-dimensional acquisition and fusion data, a road condition feature label coding vector of the sample multi-dimensional acquisition and fusion data, and a road condition severity coding vector of the sample multi-dimensional acquisition and fusion data; According to the sample road condition state trend coding vector and the road condition feature label coding vector of the sample multi-dimensional acquisition and fusion data, a first network learning task of a highway road condition prediction network is performed, wherein the first network learning task is used to perform parameter learning on the highway road condition prediction network so that the highway road condition prediction network predicts the road condition feature label corresponding to the sample multi-dimensional acquisition and fusion data; According to the sample road condition state trend coding vector and the road condition severity coding vector of the sample multi-dimensional acquisition and fusion data, the second network learning task of the highway road condition prediction network is run, and the second network learning task is used to perform parameter learning on the highway road condition prediction network so that the highway road condition prediction network predicts the road condition severity of the sample multi-dimensional acquisition and fusion data, so that the highway road condition prediction network that has completed parameter learning predicts the road condition feature label and road condition severity of any target multi-dimensional acquisition and fusion data; The road condition feature annotation data of the sample multi-dimensional acquisition and fusion data also includes road condition feature knowledge points of the sample multi-dimensional acquisition and fusion data under the road condition feature label, and each of the road condition feature labels includes a plurality of the road condition feature knowledge points; The coding vector sequence of the highway road condition learning data also includes the road condition feature knowledge point coding vector of the road condition feature knowledge point; The step of running the second network learning task of the highway road condition prediction network based on the sample road condition state trend coding vector and the road condition severity coding vector of the sample multi-dimensional acquisition fusion data includes: According to the sample road condition state trend coding vector, the road condition severity coding vector and the road condition feature knowledge point coding vector of the sample multi-dimensional acquisition and fusion data, the second network learning task of the highway road condition prediction network is run, and the second network learning task is also used to predict the road condition feature knowledge point of the sample multi-dimensional acquisition and fusion data under the road condition feature label when predicting the road condition feature label of the sample multi-dimensional acquisition and fusion data; The step of encoding each of the highway road condition learning data to generate an encoding vector sequence for each of the highway road condition learning data comprises: Traversing each of the highway road condition learning data, performing feature trajectory decomposition on the highway road condition learning data, and generating a plurality of feature trajectory segments; Replacing the road condition feature annotation data in the highway road condition learning data with a shielding placeholder symbol; Taking each of the plurality of characteristic trajectory segments as a reference coding unit, the shielding placeholder symbol and the reference coding units respectively corresponding to the plurality of characteristic trajectory segments are respectively associated according to data nodes in the highway road condition learning data to generate a to-be-coded set of the highway road condition learning data, wherein a starting coding unit is inserted at the starting end of the to-be-coded set; Performing feature mapping processing on the to-be-encoded set to generate data encoding vector distribution and node encoding vector distribution corresponding to the highway road condition learning data; Fusion is performed based on the data coding vector distribution and the node coding vector distribution to generate a coding vector sequence for the highway road condition learning data; The step of acquiring a first highway road condition learning data sequence comprises: Acquire a previously collected second highway road condition learning data sequence, wherein the second highway road condition learning data sequence includes the sample multi-dimensional acquisition fusion data and a road condition feature label and road condition severity of the sample multi-dimensional acquisition fusion data; Obtain the road condition feature knowledge point under the road condition feature label of the sample multi-dimensional acquisition fusion data; According to the road condition feature label of the sample multi-dimensional acquisition and fusion data, the road condition severity of the sample multi-dimensional acquisition and fusion data, and the road condition feature knowledge point of the sample multi-dimensional acquisition and fusion data under the road condition feature label, the road condition feature annotation data of the sample multi-dimensional acquisition and fusion data is formed; Constructing highway road condition learning data based on the sample multi-dimensional collected fusion data and the road condition feature annotation data, and constructing the first highway road condition learning data sequence based on a plurality of the highway road condition learning data; The step of obtaining the road condition feature knowledge point under the road condition feature label of the sample multi-dimensional acquisition fusion data includes: Acquire a previously configured road condition information comparison library, the road condition information comparison library including association information between different road condition feature labels, different road condition severity levels, and different road condition feature knowledge points; search the association information according to the road condition feature label and the road condition severity level of the sample multi-dimensional acquisition and fusion data, and generate the road condition feature knowledge point of the sample multi-dimensional acquisition and fusion data under the road condition feature label; The road condition feature knowledge points of the sample multi-dimensional collected fusion data are identified from a plurality of previously defined road condition feature knowledge points using a pre-trained scene recognition model.

2. The highway road condition monitoring method based on multi-dimensional data fusion according to claim 1 is characterized in that: The highway traffic condition prediction network includes: a long short-term memory sub-network, a first mapping sub-network, a self-attention sub-network, and a second mapping sub-network; The performing of the first network learning task of the highway road condition prediction network based on the sample road condition state trend coding vector and the road condition feature label coding vector of the sample multi-dimensional acquisition fusion data comprises: Run the long short-term memory sub-network to extract features based on the sample road condition state trend coding vector and the road condition feature label coding vector of the sample multi-dimensional collected fusion data to generate a first fusion coding vector; Running the first mapping subnetwork to perform mapping according to the first fusion coding vector to generate predicted road condition feature label data; Determine a first error value between the predicted road condition feature label data and the road condition feature label of the sample multi-dimensional acquisition fusion data, and optimize the network weights and bias information of the long short-term memory subnetwork and the network weights and bias information of the first mapping subnetwork based on the first error value; If the long short-term memory sub-network meets the training termination condition and the first mapping sub-network meets the training termination condition, ending the first network learning task; The step of running the second network learning task of the highway road condition prediction network based on the sample road condition state trend coding vector and the road condition severity coding vector of the sample multi-dimensional acquisition fusion data includes: Run the self-attention sub-network to extract features based on the sample road condition state trend coding vector and the road condition severity coding vector of the sample multi-dimensional collected fusion data to generate a second fusion coding vector; Running the second mapping subnetwork to perform mapping according to the second fusion coding vector to generate predicted road condition severity data; Determine a second error value between the predicted road condition severity data and the road condition severity of the sample multi-dimensional acquisition fusion data, and optimize the network weights and bias information of the self-attention subnetwork and the network weights and bias information of the second mapping subnetwork based on the second error value.

3. The highway road condition monitoring method based on multi-dimensional data fusion according to claim 2 is characterized in that: The predicted road condition feature label data includes the discrimination results of the sample multi-dimensional acquisition fusion data corresponding to the plurality of road condition feature labels; The determining of a first error value between the predicted road condition feature label data and the road condition feature label of the sample multi-dimensional acquisition fusion data includes: Traversing each of the plurality of road condition feature labels, and determining a discrimination error value between a discrimination result of the road condition feature label of the sample multi-dimensional collected fusion data and a true labeled discrimination result of the road condition feature label; A weighted result of the discrimination error value of each of the road condition feature labels is determined as a first error value between the predicted road condition feature label data and the road condition feature label of the sample multi-dimensional acquisition fusion data.

4. The highway road condition monitoring method based on multi-dimensional data fusion according to claim 3 is characterized in that: The determining of the discrimination error value between the discrimination result of the road condition feature label of the sample multi-dimensional acquisition fusion data and the real labeling discrimination result of the road condition feature label includes: Dividing the highway road condition learning data in the highway road condition learning data sequence into a plurality of training groups; Traversing the sample multi-dimensional acquisition fusion data in each of the highway road condition learning data in each of the training groups, determining a first exponential conversion value of a discrimination result of the road condition feature label of the sample multi-dimensional acquisition fusion data, and determining a first dot product result of the first exponential conversion value and a true label discrimination result of the sample multi-dimensional acquisition fusion data; Determine a first comparison weight between the first dot product result and the scale of the highway road condition learning data included in the training group, and use the first comparison weight as a discrimination error value between a discrimination result of the road condition feature label of the sample multi-dimensional acquisition fusion data and a true label discrimination result.

5. The highway road condition monitoring method based on multi-dimensional data fusion according to claim 2 is characterized in that: The highway traffic condition prediction network also includes a self-attention sub-network, a second mapping sub-network and a third mapping sub-network; The second network learning task of the highway road condition prediction network is run based on the sample road condition state trend coding vector, the road condition severity coding vector and the road condition feature knowledge point coding vector of the sample multi-dimensional acquisition fusion data, including: Run the self-attention sub-network to extract features based on the sample road condition state trend coding vector, the road condition severity coding vector and the road condition feature knowledge point coding vector of the sample multi-dimensional collected fusion data to generate a third fusion coding vector; Running the second mapping subnetwork to perform mapping according to the third fusion coding vector to generate predicted road condition severity data; Running the third mapping subnetwork to perform mapping according to the third fusion coding vector to generate road condition feature knowledge point prediction data; Determine a second error value between the predicted road condition severity data and the road condition severity of the sample multi-dimensional acquisition fusion data, and determine a third error value between the road condition feature knowledge point prediction data and the road condition feature knowledge point of the sample multi-dimensional acquisition fusion data; Performing a fusion calculation on the first error value, the second error value, and the third error value to generate a global error value; The network weights and bias information of the self-attention sub-network, the network weights and bias information of the second mapping sub-network, and the network weights and bias information of the third mapping sub-network are optimized according to the global error value.

6. The highway road condition monitoring method based on multi-dimensional data fusion according to claim 5 is characterized in that: The predicted road condition severity data includes the predicted road condition severity of the sample multi-dimensional acquisition and fusion data corresponding to the plurality of road condition feature labels, and the road condition severity of the sample multi-dimensional acquisition and fusion data includes the actual annotated road condition severity of the sample multi-dimensional acquisition and fusion data corresponding to the plurality of road condition feature labels; The determining of a second error value between the predicted road condition severity data and the road condition severity of the sample multi-dimensional acquisition fusion data comprises: Traversing each of the plurality of road condition feature labels, and dividing the highway road condition learning data in the highway road condition learning data sequence into a plurality of training groups; Traversing the sample multi-dimensional acquisition fusion data in each of the highway road condition learning data in each of the training groups, determining the error result between the predicted road condition severity data corresponding to the road condition feature label of the sample multi-dimensional acquisition fusion data and the actual annotated road condition severity corresponding to the road condition feature label; Determine a second comparison weight between the square of the error result and the size of the highway road condition learning data included in the training group, and use the second comparison weight as a road condition severity error value between the predicted road condition severity data corresponding to the road condition feature label of the sample multi-dimensional acquisition fusion data and the actual annotated road condition severity; A weighted result of the road condition severity error value of each road condition feature label is determined as a second error value between the predicted road condition severity data and the road condition severity of the sample multi-dimensional acquisition fusion data.

7. The highway road condition monitoring method based on multi-dimensional data fusion according to claim 5 is characterized in that: The road condition feature knowledge point prediction data includes the discrimination results of the sample multi-dimensional acquisition fusion data corresponding to the plurality of road condition feature knowledge points; The determining of a third error value between the road condition feature knowledge point prediction data and the road condition feature knowledge point of the sample multi-dimensional acquisition fusion data includes: Traversing each of the plurality of road condition feature knowledge points, and determining a knowledge point discrimination error value between a discrimination result of the road condition feature knowledge point and a true labeled discrimination result of the road condition feature knowledge point of the sample multi-dimensional collected fusion data; Determine a weighted result of the knowledge point discrimination error value of each of the road condition feature knowledge points to serve as a third error value between the road condition feature knowledge point prediction data and the road condition feature knowledge point of the sample multi-dimensional acquisition fusion data; Wherein, the determining of the knowledge point discrimination error value between the discrimination result of the road condition feature knowledge point and the real labeling discrimination result of the road condition feature knowledge point of the sample multi-dimensional acquisition fusion data includes: Dividing the highway road condition learning data in the highway road condition learning data sequence into a plurality of training groups; Traversing the sample multi-dimensional acquisition fusion data in each of the highway road condition learning data in each of the training groups, determining a second exponential conversion value of the discrimination result of the road condition feature knowledge point, and determining a second dot product result of the second exponential conversion value and the real annotation discrimination result of the road condition feature knowledge point; determining a third comparison weight between the second dot product result and the size of the highway road condition learning data included in the training group; The inverse of the third comparison weight is used as the knowledge point discrimination error value between the discrimination result of the road condition feature knowledge point and the real labeled discrimination result of the road condition feature knowledge point of the sample multi-dimensional acquisition fusion data.

8. A cloud computing system, characterized in that: The cloud computing system includes a processor and a memory, the memory is connected to the processor, the memory is used to store programs, instructions or codes, and the processor is used to run the programs, instructions or codes in the memory to implement the highway road condition monitoring method based on multidimensional data fusion as described in any one of claims 1 to 7 above.

Images