Intelligent traffic information collection method and system based on big data

Abstract

The present application provides a method and system for collecting intelligent traffic information based on big data, which can reduce the amount of data processing by screening the historical congestion sensor data set sequence of the target road to obtain an approximate data set sequence of the target road sensor data set. The target machine learning algorithm is tuned according to the historical congestion sensor data set sequence of the example road, and the data set sequence can be accurately characterized. The congestion situation of the target road can be accurately characterized to help timely traffic diversion intervention. Passenger and freight transport vehicles can also be safely managed and controlled, and the driver behavior data and vehicle driving data obtained by the on-board satellite positioning active safety terminal can be analyzed and processed to judge the driver's behavior habits, improve the driver's safe driving ability, and improve road safety. Through the Beidou satellite positioning system, in conjunction with machine learning or algorithms, the driver's driving behavior can be analyzed, the vehicle's safe driving ability can be improved, and the safety of vehicles and roads can be improved.

Description

Technical Field

[0001] The present application relates to but is not limited to the field of data processing technology, and in particular to a method and system for collecting smart traffic information based on big data. Background Art

[0002] With the acceleration of urbanization, urban traffic problems have become increasingly prominent, especially traffic congestion has become an important factor restricting urban development. In order to effectively alleviate traffic congestion and improve road traffic efficiency, smart transportation systems have emerged. Smart transportation systems integrate advanced information technology, data communication transmission technology, electronic sensor technology, control technology and computer technology to achieve comprehensive perception, intelligent analysis and active management of traffic. In smart transportation systems, the collection and analysis of traffic data can be performed in parallel to ensure the timeliness of congestion warning. Traditional traffic data collection methods often rely on manual statistics or fixed-point camera monitoring. These data not only have a long collection cycle and high cost, but also are difficult to fully and real-time reflect changes in traffic conditions. With the development of Internet of Things technology, traffic sensor networks have gradually become an important means of traffic data collection. By deploying traffic sensor networks on target roads, various traffic parameters such as vehicle speed, traffic flow, lane occupancy, etc. can be collected in real time and accurately, providing strong support for the decision-making of smart transportation systems.

[0003] However, facing the massive amount of traffic data, how to analyze and process it efficiently and accurately has become a major challenge facing the intelligent transportation system. Traditional data analysis methods often require comprehensive processing of all data, which is not only computationally intensive and time-consuming, but also easily introduces noise and redundant information, affecting the accuracy of the analysis results. Therefore, how to improve data processing efficiency while ensuring the accuracy of the analysis results has become an urgent problem to be solved in the intelligent transportation system. Summary of the invention

[0004] In view of this, the present application provides a method and system for collecting intelligent traffic information based on big data. The technical solution of the present application is implemented as follows:

[0005] On the one hand, the present application provides a method for collecting smart traffic information based on big data, the method comprising: collecting a target road sensor data set based on a traffic sensor network deployed on a target road, and obtaining a historical congestion sensor data set sequence of the target road; determining an approximate data set sequence of the target road sensor data set from the historical congestion sensor data set sequence of the target road; obtaining a collaborative data set sequence based on the approximate data set sequence and the target road sensor data set, and passing the collaborative data set sequence to a target machine learning algorithm; the target machine learning algorithm includes a priori sensor data descriptor set, and the target machine learning algorithm is tuned according to the historical congestion sensor data set sequence of an example road; using the target machine learning algorithm, in the priori sensor data descriptor set, indexing the priori sensor data descriptor corresponding to each sensor data set in the collaborative data set sequence, integrating the descriptor information of each priori sensor data descriptor obtained by the index, and obtaining a data sequence descriptor of the collaborative data set sequence, and the data sequence descriptor is used to determine the confidence of the target road sensor data set in representing the target road congestion warning.

[0006] On the other hand, the present application provides a computer system, including a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor implements the steps in the above method when executing the program.

[0007] The beneficial effects of the present application include at least: the present application collects the target road sensor data set based on the traffic sensor network deployed on the target road, obtains the historical congestion sensor data set sequence of the target road, determines the approximate data set sequence of the target road sensor data set from the historical congestion sensor data set sequence of the target road, obtains the collaborative data set sequence according to the approximate data set sequence and the target road sensor data set, and transmits the collaborative data set sequence to the target machine learning algorithm, the target machine learning algorithm includes a priori sensor data descriptor set, the target machine learning algorithm is tuned according to the historical congestion sensor data set sequence of the example road, and adopts the target machine learning algorithm. In the priori sensor data descriptor set, the priori sensor data descriptor corresponding to each sensor data set in the collaborative data set sequence is indexed, and the descriptor information of each priori sensor data descriptor obtained by the index is integrated to obtain the data sequence descriptor of the collaborative data set sequence, and the data sequence descriptor is used to determine the confidence of the target road sensor data set characterizing the congestion warning of the target road. In this way, the historical congestion sensor data set sequence of the target road is screened to obtain the approximate data set sequence of the target road sensor data set, which can reduce the amount of data processing and improve the algorithm speed. The target machine learning algorithm is tuned based on the historical congestion sensor dataset sequence of the example road, and can accurately characterize the dataset sequence. The collaborative dataset sequence obtained based on the approximate dataset sequence and the target road sensor dataset is passed to the target machine learning algorithm for processing, and the obtained characterization result is accurate and reliable. The target machine learning algorithm uses the original sensor dataset descriptor obtained by indexing the target road sensor dataset and the approximate dataset sequence to perform collaborative characterization modeling on the target road sensor dataset and the approximate dataset sequence. The obtained data sequence descriptor can accurately characterize the congestion situation of the target road based on the corresponding target road sensor dataset. The confidence of the target road congestion warning characterized by the target road sensor dataset is determined based on the data sequence descriptor, which can help to carry out timely traffic diversion intervention. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0009] Figure 1 A schematic diagram of the implementation process of a smart traffic information collection method based on big data provided in an embodiment of the present application.

[0010] Figure 2 A hardware entity diagram of a computer system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0011] The embodiment of the present application provides a method for collecting intelligent traffic information based on big data, which can be executed by a processor of a computer system, wherein the computer system can refer to a device with data processing capabilities such as a server, a laptop, a tablet computer, and a desktop computer.

[0012] Figure 1 A schematic diagram of the implementation process of a smart traffic information collection method based on big data provided in the embodiment of the present application is shown in FIG. Figure 1 As shown, the method includes:

[0013] Step S100: collecting a target road sensor data set based on a traffic sensor network deployed on the target road, and obtaining a historical congestion sensor data set sequence of the target road.

[0014] In an actual application scenario example, an advanced traffic sensor network is installed on the main roads of a large city. These sensors are carefully arranged at various key locations on the road, including but not limited to intersections, bridges, tunnels, and congested sections. These sensors are of various types, including but not limited to geomagnetic sensors, microwave radars, infrared sensors, and high-definition cameras, each of which is good at capturing different types of traffic data.

[0015] Geomagnetic sensors: buried under the road surface, can accurately measure the number, speed and type of vehicles passing by (such as small cars, large cars, etc.). The datasets they generate may contain features such as timestamp, lane number, vehicle count and average speed. For example: timestamp = "2023-04-01 15:00:00", lane number = 1, vehicle count = 30, average speed = 40km / h.

[0016] Microwave radar: Installed above the road, it detects the presence, speed, and direction of vehicles by emitting microwaves and receiving reflected signals. Its data set may include the vehicle's position coordinates, velocity vector, and direction of travel. For example: vehicle position = (x = 100m, y = 5m), velocity vector = (vx = 30m / s, vy = 0m / s), and direction of travel = east.

[0017] Infrared sensor: used to detect the dynamics of pedestrians and non-motor vehicles, especially on sidewalks and bicycle lanes. Its data set contains the flow of pedestrians or non-motor vehicles, passing time, etc. For example: pedestrian flow = 5, passing time = "2023-04-01 15:00:00 to 15:05:00".

[0018] High-definition cameras: They not only monitor traffic conditions, but also analyze vehicle types, license plate numbers, and even driver behavior through image recognition technology. Their data sets exist in the form of images or video streams, but can also be converted into structured data after processing, such as the number of vehicles of a specific type in a specific time period.

[0019] By integrating the outputs of these sensors, the sensor data set of the target road is constructed in real time. Each data set is a multidimensional vector or a structured data record that contains the various features mentioned above. At the same time, the system will retrieve the historical congestion sensor data set sequence of the same target road from the historical database. These historical data are arranged in chronological order and record the occurrence, duration, impact range and various traffic parameters related to road congestion in the past period of time.

[0020] Step S200: determining an approximate data set sequence of a target road sensor data set from a historical congestion sensor data set sequence of the target road.

[0021] In step S200, the computer system accurately identifies a data set sequence that is most similar to the current sensor data set of the target road from the historical congestion sensor data set sequence, that is, an approximate data set sequence.

[0022] For example, suppose the target road is a busy urban arterial road, and the historical congestion sensor dataset sequence of the road has been exhaustively recorded and stored in the database of the computer system. Each historical congestion sensor dataset contains multiple key features, such as congestion level, average vehicle speed, traffic volume, congestion duration, etc. These features together constitute a multi-dimensional description of past congestion conditions.

[0023] An example of a current sensing data set for a target road may be: timestamp: 2023-04-02 15:00:00; average vehicle speed: 35km / h; vehicle volume: 1200 vehicles / hour; lane occupancy rate: 60%…

[0024] To find a historical data set that is similar to the current sensor data set for the target road, the computer system performs the following steps:

[0025] Feature extraction and dimensionality reduction: First, the system extracts key features from the current sensor data set of the target road and aligns them with the corresponding features in the historical congestion sensor data set. Since the original feature set may be very large and contain redundant information, the system may use feature dimensionality reduction techniques (such as principal component analysis PCA, t-SNE, etc.) to reduce the feature dimension while trying to retain the key information in the data.

[0026] Example after feature dimensionality reduction: Dimensionality reduction feature 1 (comprehensive vehicle speed and flow): 0.5 (standardized value); Dimensionality reduction feature 2 (lane occupancy): 0.6 (standardized value)...

[0027] Similarity measurement: Next, the system calculates the similarity between the reduced-dimensional feature vector of the current sensor dataset of the target road and the reduced-dimensional feature vector of each dataset in the historical congestion sensor dataset sequence. Commonly used similarity measurement methods include cosine similarity, Euclidean distance, etc.

[0028] Similarity calculation example (using cosine similarity): Similarity = A·B / |A| |B|.

[0029] Among them, A and B represent the reduced dimension feature vectors of the current sensor dataset and the historical congestion sensor dataset of the target road, respectively.

[0030] Determine the approximate dataset sequence: Based on the results of the similarity measurement, the system selects several datasets with the highest similarity to the current sensor dataset of the target road from the historical congestion sensor dataset sequence. These datasets are arranged in the chronological order of their occurrence in the historical records to form an approximate dataset sequence. An example of an approximate dataset sequence is: timestamp 1: 2023-03-28 15:10:00, similarity: 0.92; timestamp 2: 2023-03-30 14:50:00, similarity: 0.88; timestamp 3: 2023-04-01 15:05:00, similarity: 0.90...

[0031] These approximate datasets not only provide historical congestion patterns similar to the current traffic conditions of the target roads, but also reveal the evolution of these patterns over time. By comparing and analyzing these approximate datasets, the computer system can more accurately assess the development trend of the current traffic conditions and predict whether congestion is likely to occur in the future.

[0032] Step S300: A collaborative dataset sequence is obtained based on the approximate dataset sequence and the target road sensor dataset, and the collaborative dataset sequence is passed to the target machine learning algorithm; the target machine learning algorithm includes a priori sensor data descriptor set, and the target machine learning algorithm is tuned based on the historical congestion sensor dataset sequence of the example road. Among them, the priori sensor data descriptor in the priori sensor data descriptor set is the target sensor dataset descriptor of the sensor dataset, and the target sensor dataset descriptor of the sensor dataset is obtained by performing a descriptor dimension compression operation on the original sensor dataset descriptor of the sensor dataset.

[0033] In step S300, the computer system integrates the current sensor data set of the target road and the approximate data set sequence selected from historical data, constructs a collaborative data set sequence, and inputs this sequence into a pre-trained target machine learning algorithm for analysis.

[0034] First, the computer system merges the current sensor data set of the target road with the sequence of approximate data sets to form a collaborative data set sequence. This sequence not only contains the current real-time traffic parameters (such as vehicle speed, traffic volume, lane occupancy, etc.), but also incorporates detailed records of similar traffic conditions in history. Each data set is arranged in chronological order in the collaborative sequence, forming a coherent data stream that reflects the changing trend of traffic conditions from the past to the present. As an example of a collaborative dataset sequence: Timestamp 1 (historical): 2023-03-30 14:50:00, vehicle speed = 30km / h, vehicle flow = 1000 vehicles / hour (from the approximate dataset); Timestamp 2 (historical): 2023-03-30 15:00:00, vehicle speed = 25km / h, vehicle flow = 1100 vehicles / hour (from the approximate dataset); ... (more historical data points); Timestamp N (current): 2023-04-02 15:00:00, vehicle speed = 35km / h, vehicle flow = 1200 vehicles / hour (from the target road sensing dataset).

[0035] Next, the computer system inputs the constructed collaborative dataset sequence into the target machine learning algorithm. This algorithm is specially designed for intelligent traffic congestion warning. It has been tuned with a large number of historical congestion sensor dataset sequences of example roads (i.e., the roads used when the algorithm was trained) and has learned how to identify the precursor features of traffic congestion and predict future congestion trends. The target machine learning algorithm contains a set of prior sensor data descriptors, which are abstract representations of key features in historical congestion data and can capture the essential laws of traffic state changes. The algorithm compares the real-time data in the collaborative dataset sequence with the prior descriptors to find the match and correlation between them, and then infers whether the current traffic situation is likely to evolve into congestion.

[0036] The target machine learning algorithm may adopt an architecture based on a recurrent neural network (RNN) or its variants (such as LSTM, GRU), which are good at processing sequence data and can capture the time dependency in the data. The training process of the algorithm may involve the following key steps: Feature extraction: Extract key features such as vehicle speed, traffic volume, lane occupancy, etc. from the historical congestion sensor data set and construct them into feature vectors. Sequence modeling: Use the RNN network to model the feature vector sequence and learn the pattern of traffic state changes over time. Loss function calculation: Define a loss function (such as mean square error MSE or cross entropy loss) to measure the difference between the model prediction results and the actual congestion situation. Parameter optimization: Adjust the weight parameters of the RNN network through the back propagation algorithm to minimize the loss function and improve the prediction accuracy of the model. Model evaluation and tuning: Evaluate the model performance on an independent validation set, and further tune the model according to the evaluation results until a satisfactory prediction effect is achieved. The specific training process of the algorithm will be introduced later in this application, which will not be repeated here.

[0037] Step S400: Using the target machine learning algorithm, in the prior sensor data descriptor set, index the prior sensor data descriptor corresponding to each sensor data set in the collaborative data set sequence, integrate the descriptor information of each prior sensor data descriptor obtained by the index, and obtain the data sequence descriptor of the collaborative data set sequence. The data sequence descriptor is used to determine the confidence level of the target road sensor data set in representing the target road congestion warning.

[0038] In step S400, the computer system uses the target machine learning algorithm to perform an index operation in a pre-constructed set of prior sensor data descriptors. These prior descriptors are highly abstract features of a large amount of historical congestion data and can represent different types of traffic congestion patterns. For each sensor data set in the collaborative data set sequence (whether historical data or current real-time data), the system will find the prior descriptor that best matches it. For example, for the current real-time sensor data set (timestamp 2023-04-02 15:05:00, vehicle speed = 33km / h, vehicle flow = 1300 vehicles / hour), the system may index to a prior sensor data descriptor indicating that "mild congestion is beginning to form".

[0039] After obtaining the prior descriptors corresponding to all sensor data sets, the system enters the descriptor information integration stage. This process is essentially a deep interaction and fusion of traffic state features at different time points, aiming to capture the complex dynamics of traffic state evolution over time. Integration methods may include but are not limited to feature splicing, attention mechanism weighting, graph neural network propagation, etc. Taking feature splicing as an example, the system will splice all prior descriptors into a long vector according to their time order in the collaborative data set sequence. This long vector is the data sequence descriptor. This descriptor not only contains the key traffic features at each time point, but also implies the temporal dependencies and mutual influences between these features.

[0040] The resulting data sequence descriptor is used to determine the confidence level of the target road sensor data set in representing the target road congestion warning. The system may input the data sequence descriptor into a trained classifier or regressor, which can output a value representing the confidence level of the congestion warning based on the feature combination of the descriptor. For example, if the classifier outputs a value close to 1, the system can believe with high confidence that the target road is about to or is experiencing congestion; conversely, if the output value is low, it indicates that the current traffic conditions are relatively smooth.

[0041] In a feasible implementation scheme, step S200, determining an approximate data set sequence of a target road sensor data set from a historical congestion sensor data set sequence of the target road, includes:

[0042] Step S210: extracting a target sensor data set descriptor of the sensor data set for the target road and the historical congestion sensor data set in the historical congestion sensor data set sequence of the target road; wherein the target sensor data set descriptor is obtained by performing a descriptor dimension compression operation on the original sensor data set descriptor of the sensor data set;

[0043] Step S220: determining an approximate data set sequence of the target road sensor data set from the historical congestion sensor data set sequence of the target road according to a similarity measure between the target sensor data set descriptors of the target road sensor data set and the historical congestion sensor data set.

[0044] In step S210, the computer system extracts the target sensor data set descriptors of each data set in the target road sensor data set and its historical congestion sensor data set sequence. These descriptors are highly abstract of the data set features, which can effectively represent the core information of the data set while reducing the computational complexity and storage requirements. The sensor data set usually contains multiple types of traffic parameters, such as vehicle speed, vehicle flow, lane occupancy, vehicle type distribution, etc. These parameters constitute the original feature set of the sensor data set. The computer system first needs to select the most critical features for congestion warning from the original feature set based on domain knowledge and data characteristics as the basis for subsequent analysis. For example, for the two basic features of vehicle speed and vehicle flow, they directly reflect the road's capacity and traffic pressure. The computer system may select these two features, as well as their combination (such as the ratio of average vehicle speed to vehicle flow) as components of the original sensor data set descriptors.

[0045] Since the original feature set may be very large, directly using it for similarity calculation is not only inefficient, but may also introduce unnecessary noise. Therefore, the computer system uses descriptor dimension compression technology to reduce the dimension of the original sensor data set descriptor to obtain the target sensor data set descriptor.

[0046] Commonly used feature dimensionality reduction methods include principal component analysis (PCA), linear discriminant analysis (LDA), t-distributed stochastic neighbor embedding (t-SNE), etc. Taking PCA as an example, this method converts the original features that may be related into a set of linearly unrelated features, namely principal components, through orthogonal transformation. These principal components are arranged from large to small according to variance, and the computer system can select the first few principal components with the largest variance as the descriptors of the target sensor data set, thereby achieving dimensionality reduction.

[0047] For example, assuming that the original sensor data set descriptor contains three features: vehicle speed (v), traffic flow (f), and lane occupancy (o), it can be represented as a three-dimensional vector: [v, f, o]. After PCA dimensionality reduction, assuming that the first two principal components are selected as the target sensor data set descriptor, the reduced descriptor can be represented as a two-dimensional vector: [PC1, PC2]. Among them, PC1 and PC2 are the first and second principal components of the original features after PCA transformation.

[0048] After obtaining the target sensor dataset descriptor of each dataset in the target road sensor dataset and its historical congestion sensor dataset sequence, the computer system next needs to calculate the similarity between these descriptors and filter out the approximate dataset sequence based on the similarity measurement results.

[0049] Similarity measurement is a key step in evaluating the similarity between two data sets. In the embodiments of the present application, commonly used similarity measurement methods include cosine similarity, Euclidean distance, Manhattan distance, etc. Among them, cosine similarity is widely used in text processing, image processing, and high-dimensional data analysis because it can ignore the vector length (i.e., the absolute size of the eigenvalue) and only focus on the vector direction (i.e., the relative relationship between the features). For two target sensor data set descriptor vectors A and B, the cosine similarity between them can be calculated by the following formula:

[0050] ;

[0051] Among them, A i and B i are the i-th components of vectors A and B respectively, and n is the dimension of the vector.

[0052] The computer system traverses the sequence of historical congestion sensor data sets of the target road, calculates the similarity between the target sensor data set descriptor of the current sensor data set and the target sensor data set descriptor of each historical congestion sensor data set in the sequence, and then sorts the historical congestion sensor data sets from high to low according to the similarity measurement result.

[0053] In order to determine the final length of the approximate dataset sequence, the computer system can use a preset threshold method or a fixed number method. The preset threshold method refers to setting a similarity threshold, and only historical congestion sensor datasets with a similarity higher than the threshold will be selected into the approximate dataset sequence; the fixed number rule refers to directly selecting the top N historical congestion sensor datasets with the highest similarity as the approximate dataset sequence. For example, suppose the target sensor dataset descriptor of the target road sensor dataset is: [0.5, 0.6]. The target sensor dataset descriptor of a historical congestion sensor dataset in the historical congestion sensor dataset sequence is: [0.45, 0.65]. The similarity between them calculated by the cosine similarity formula is 0.97.

[0054] If the preset similarity threshold is 0.9, then historical descriptor 1 will be selected into the approximate dataset sequence. By repeating the above process, the computer system can construct an approximate dataset sequence containing multiple historical congestion sensor datasets. This sequence not only reflects the similarity between the current traffic state of the target road and the historical congestion pattern, but also provides valuable data support for subsequent congestion warning analysis.

[0055] In summary, step S200 successfully selects the approximate data set sequence that is most similar to the current sensor data set of the target road from the rich historical congestion sensor data set sequence of the target road through a series of fine operations such as feature selection, dimensionality reduction, and similarity measurement. This process not only reflects the powerful ability of computer systems in processing large-scale, high-dimensional data, but also lays a solid foundation for the further development of intelligent traffic information collection and congestion warning systems.

[0056] In a feasible implementation scheme, step S210, for the target road sensor data set and the historical congestion sensor data set in the sequence of historical congestion sensor data sets of the target road, extracting a target sensor data set descriptor of the sensor data set includes:

[0057] Step S211: obtaining a target data set label stream according to the data set label formation of the current sensor data set, and inputting the target data set label stream into the target descriptor dimension compression algorithm; the target data set is a target road sensor data set or a historical congestion sensor data set in a sequence of historical congestion sensor data sets of the target road, and the target descriptor dimension compression algorithm includes an original sensor data set descriptor set;

[0058] Step S212: Using a target descriptor dimension compression algorithm, in the original sensor data set descriptor set, index and obtain the original sensor data set descriptor corresponding to the data set label of the current sensor data set, highlight the descriptor information and compress the descriptor dimension of the indexed original sensor data set descriptor, and obtain the target sensor data set descriptor of the current sensor data set.

[0059] In step S211, the computer system first focuses on the current sensor data set and its counterpart in the historical congestion sensor data set sequence, and constructs a target data set label stream by integrating their data set labels. Data set labels are key identifiers that distinguish different data sets, and they may contain multiple attributes such as timestamps, location information, and traffic condition categories. In the embodiment of the present application, the timestamp is particularly important because it is directly related to the trend of traffic conditions over time.

[0060] Assuming that the dataset label of the target road sensor dataset contains a timestamp and a preliminary judgment of the traffic conditions (such as unblocked, light congestion, moderate congestion, and heavy congestion), a typical dataset label may be in the form of "2023-04-02 15:00:00, light congestion". For the historical congestion sensor dataset sequence, each historical congestion sensor dataset also has a similar dataset label.

[0061] The computer system concatenates these dataset labels in chronological order to form a target dataset label stream. For example, for the current sensing dataset and its historical congestion sensing datasets a few hours before, the target dataset label stream may be as follows: ["2023-04-02 14:30:00, unblocked", "2023-04-02 14:45:00, light congestion", "2023-04-02 15:00:00, light congestion"].

[0062] Next, the computer system feeds the target dataset label stream into the target descriptor dimension compression algorithm. The algorithm is a pre-designed and trained model or algorithm framework that aims to extract and compress more representative target sensor dataset descriptors from the original sensor dataset descriptor set. The original sensor dataset descriptor set is a database containing a large number of historical sensor dataset feature vectors that comprehensively cover possible traffic condition characteristics.

[0063] In step S212, the computer system uses the target descriptor dimension compression algorithm to perform a series of complex operations on each dataset tag in the target dataset tag stream to obtain the corresponding target sensor dataset descriptor. First, the algorithm performs an index operation in the original sensor dataset descriptor set according to the dataset tag of the current sensor dataset (or the historical congestion sensor dataset). This step is similar to querying a specific record in a database, and its purpose is to find the original sensor dataset descriptor that matches the current dataset tag. Assuming that the dataset tag of the current sensor dataset is "2023-04-02 15:00:00, light congestion", the algorithm will search for all descriptors in the original sensor dataset descriptor set whose tags contain "2023-04-02 15:00:00" and the traffic condition is initially judged as "light congestion". Since there may be multiple fully matched or approximately matched descriptors, the algorithm may need to further filter according to conditions such as time accuracy and location similarity.

[0064] After indexing the original sensor dataset descriptor, the algorithm then performs a descriptor information highlighting operation, i.e., feature enhancement. The purpose of this step is to enhance the key feature semantic information in the descriptor to make it more prominent and easier to analyze later. Feature enhancement can be achieved in a variety of ways, including but not limited to transformer-based attention mechanisms, CNN-based convolution operations, etc.

[0065] Taking transformer as an example, the algorithm can treat the original sensor data set descriptor as a sequence, and calculate the association weights between each feature in the sequence and other features through the self-attention mechanism. These weights reflect the degree of mutual influence between features, and the algorithm can perform weighted summation or reordering of features based on these weights, thereby highlighting key features and suppressing non-key features.

[0066] After feature enhancement, the feature dimension of the original sensor data set descriptor may still be high, which is not conducive to subsequent calculation and storage. Therefore, the last step of the algorithm is to compress the descriptor dimension. Dimension compression can be achieved through traditional methods such as PCA and LDA, or deep learning models (such as autoencoders) can be used for more complex nonlinear dimensionality reduction. Taking PCA as an example, the algorithm will perform an orthogonal transformation on the descriptor after feature enhancement to obtain a set of linearly uncorrelated principal components. Then, the algorithm selects the first few principal components with the largest variance as the target sensor data set descriptors, thereby achieving dimensional compression. These principal components not only retain most of the information in the original descriptors, but also greatly reduce the redundancy of the data and the computational complexity.

[0067] For example, suppose the original sensor data set descriptor is a vector containing 100 features. After feature enhancement and PCA dimensionality reduction, the target sensor data set descriptor becomes a vector containing 10 principal components. These principal components may represent the combined features of key traffic parameters such as vehicle speed, traffic volume, and lane occupancy, which can more effectively reflect the essential characteristics of the current traffic conditions.

[0068] In summary, step S210 extracts the target sensor dataset descriptor from the target road sensor dataset and its historical congestion sensor dataset sequence by forming a target dataset label stream, indexing the original sensor dataset descriptor, and performing a series of operations such as descriptor information highlighting and dimension compression. These descriptors not only have lower dimensions and higher information density, but also fully retain the key feature information in the original dataset, providing strong support for subsequent similarity measurement and congestion warning analysis.

[0069] In a feasible implementation scheme, the method also includes a training process of the target descriptor dimension compression algorithm, specifically including:

[0070] Step S21a: Obtain a first data set label stream, and input the first data set label stream into an original descriptor dimension compression algorithm; the first data set label stream is obtained based on a historical congestion sensor data set sequence of a first example road, and the original descriptor dimension compression algorithm includes an original sensor data set descriptor set.

[0071] In step S21a, the computer system completes the preparation of data and starts the training process of the target descriptor dimension compression algorithm. This process is the basic link in the intelligent transportation information collection and analysis framework.

[0072] First, the computer system focuses on the first example road, which is a road with rich historical data and representative traffic conditions. By deeply mining the historical congestion sensor data set sequence of this road, the computer system is able to obtain a series of time series data on traffic conditions. These data not only record basic information such as traffic flow, vehicle speed, lane occupancy rate at different time points, but also imply complex dynamic processes such as the formation, evolution and dissipation of traffic congestion. In order to convert these raw data into the format required for algorithm training, the computer system performs the following operations:

[0073] Dataset label generation: For each time point in the historical congestion sensor dataset sequence, the computer system generates a dataset label based on its corresponding traffic conditions (such as unblocked, light congestion, moderate congestion, heavy congestion, etc.). These labels not only contain timestamp information, but also clearly identify the level of traffic congestion at that time point. For example, a typical dataset label may be in the form of "2023-03-28 08:00:00, unblocked".

[0074] First dataset label stream construction: Subsequently, the computer system concatenates these dataset labels in chronological order to form an ordered dataset label stream, namely the first dataset label stream. This label stream fully reflects the traffic condition changes of the first example road over the past period of time and is an important input data for subsequent algorithm training.

[0075] After preparing the first dataset label stream, the computer system passes it to the original descriptor dimension compression algorithm. This algorithm is a pre-designed framework that aims to extract low-dimensional and key information-rich target sensor dataset descriptors from high-dimensional original sensor dataset descriptors. The original descriptor dimension compression algorithm contains a set of original sensor dataset descriptors, which is a huge database that stores feature vectors extracted from a large number of historical sensor datasets. These feature vectors cover possible traffic condition features, such as vehicle speed, traffic volume, lane occupancy, vehicle type distribution, etc. Each feature vector corresponds to a specific dataset label, representing the traffic conditions at that point in time.

[0076] In step S21a, the first dataset label stream is passed as input data of the algorithm. The algorithm will index the corresponding original sensor dataset descriptor in the original sensor dataset descriptor set according to each dataset label in the label stream. These original descriptors will become the basis for subsequent descriptor information highlighting and dimension compression operations.

[0077] Assume that the first example road is a main road connecting the city center and the suburbs, which is often congested during the morning and evening rush hours on weekdays. Through the analysis of the historical congestion sensor dataset sequence, the computer system generates a first dataset label stream, which contains dataset labels at multiple time points, such as "2023-03-28 07:30:00, unblocked", "2023-03-28 08:00:00, light congestion", "2023-03-28 08:30:00, moderate congestion", etc. These labels are then passed into the original descriptor dimension compression algorithm. The algorithm indexes the original sensor dataset descriptor corresponding to each label in the original sensor dataset descriptor set, which may contain values ​​of multiple features such as vehicle speed, traffic volume, lane occupancy, etc. For example, for the label "2023-03-28 08:00:00, mild congestion", the corresponding original sensor data set descriptor may be in the form of [v=40km / h, f=800 vehicles / hour, o=60%], where v represents vehicle speed, f represents traffic flow, and o represents lane occupancy.

[0078] Step S21b: adopting the original descriptor dimension compression algorithm, in the original sensor data set descriptor set, index and obtain the original sensor data set descriptor corresponding to each data set label before the first sequence in the first data set label stream, highlight the descriptor information and compress the descriptor dimension of each original sensor data set descriptor obtained by the index, and obtain the inference sensor data set descriptor corresponding to the first sequence, and obtain the first inference support coefficient corresponding to the data set label at the first sequence in the first data set label stream according to the inference sensor data set descriptor corresponding to the first sequence; the first sequence is determined in each data set label sequence of the first data set label stream, and the first inference support coefficient represents the inference correlation between the sensor data set corresponding to the first sequence and the forward data set sequence of the first sequence.

[0079] In step S21b, the computer system deeply applies the original descriptor dimension compression algorithm, and through a series of complex processing steps, extracts and compresses key information from the original sensor data set descriptor set, and then calculates the first reasoning support coefficient corresponding to each data set label in the first data set label stream.

[0080] Assume that the first dataset label stream is generated based on a historical congestion sensor dataset sequence of a busy urban trunk road (i.e., the first example road), which contains traffic condition labels at multiple time points, such as "2023-04-01 07:30:00, smooth", "2023-04-01 08:00:00, light congestion", etc. These labels are arranged in chronological order, reflecting the dynamic changes of road traffic conditions over time.

[0081] In step S21b, the computer system first determines the first order, that is, the position of the dataset label currently being processed in the label stream. Taking the second dataset label order as an example (assuming it is "2023-04-01 08:00:00, light congestion"), the system needs to pay attention to the dataset label before this label (i.e., the first order) and its corresponding original sensor dataset descriptor.

[0082] The computer system indexes the corresponding original sensor data set descriptor according to the first data set label (i.e., "2023-04-01 07:30:00, unblocked") in the original sensor data set descriptor set. This descriptor may be a multi-dimensional vector containing the values ​​of multiple features such as vehicle speed, traffic volume, lane occupancy, etc.

[0083] Next, the system performs information highlighting processing on the indexed original sensor dataset descriptor, that is, enhancing the semantic information of key features. This step may involve feature transformation, weighting, or feature enhancement technology based on deep learning models (such as the attention mechanism of transformer). After that, the system performs descriptor dimension compression to reduce the number of features and retain key information. The compressed descriptor is the inference sensor dataset descriptor corresponding to the first dataset label.

[0084] With the inference sensor dataset descriptor, calculate the first inference support coefficient corresponding to the current dataset label (i.e., the second dataset label). This coefficient reflects the probability or confidence of inferring the traffic conditions at the current time point (i.e., the dataset to which the second dataset label belongs) based on the traffic conditions at the previous time point (inference sensor dataset descriptor_1). The calculation process may involve complex probabilistic models or machine learning classifiers, such as softmax regression models. The model outputs a probability distribution based on the inference sensor dataset descriptor_1, and then selects the probability value corresponding to the current dataset label as the first inference support coefficient.

[0085] If the output of the softmax regression model is a vector containing multiple category probabilities, and the category probability corresponding to the current dataset label is 0.8, then the 0.8 is the first inference support coefficient.

[0086] Through this process, the computer system not only extracts and compresses key traffic feature information, but also quantifies the inference correlation between traffic conditions at different time points. This correlation is of great significance for understanding the formation mechanism of traffic congestion, predicting future traffic conditions, and formulating effective traffic management strategies. In the scenario of intelligent traffic information collection and analysis, step S21b is one of the key steps to achieve these goals.

[0087] Step S21c: Obtain a first tuning cost value according to first reasoning support coefficients corresponding to the dataset labels at each first sequence in the first dataset label stream.

[0088] In step S21c, the computer system calculates a first tuning cost value using the first reasoning support coefficient corresponding to each first-order dataset label in the first dataset label stream. The tuning cost value is an important indicator for measuring the gap between the current performance of the algorithm and the ideal performance, and it guides the optimization direction of the algorithm parameters.

[0089] In an embodiment of the present application, the first data set label stream records the time series information of the historical congestion sensor data set sequence of a certain road (such as the first example road), and each label represents the traffic conditions at a specific time point. Through step S21b, the computer system has calculated the first reasoning support coefficients corresponding to each of these labels, which reflect the confidence or probability of reasoning about the current traffic conditions based on historical data. In step S21c, the computer system needs to integrate these first reasoning support coefficients to evaluate the accuracy of the entire reasoning process. This evaluation is achieved by calculating the first tuning cost value, which is usually obtained by comparing the difference between the reasoning support coefficient and the true label (or target value).

[0090] Assume that the cross entropy loss function is used to calculate the first tuning cost value, which is one of the commonly used loss functions in classification problems. The cross entropy loss measures the difference between two probability distributions, one of which is the predicted distribution of the model (i.e., the first inference support coefficient) and the other is the true label distribution (in this case, since each label is unique, the true label distribution can be simplified to a one-hot vector, where only the position corresponding to the label is 1 and the rest are 0).

[0091] For each first sequence i in the label stream of the first dataset, assuming there are C possible traffic condition categories (such as smooth, light congestion, moderate congestion, heavy congestion, etc.), the first inference support coefficient predicted by the model can be expressed as a C-dimensional vector (y_i = [y_{i1}, y_{i2}, ..., y_{iC}]), where (y_{ic}) represents the probability that the i-th label belongs to the c-th category. The true label is a one-hot vector (t_i = [t_{i1}, t_{i2}, ..., t_{iC}]), where only the position corresponding to the true category is 1, and the rest are 0.

[0092] The calculation formula of the cross entropy loss function for a single sample is:

[0093] ;

[0094] Since the true label is a one-hot vector, the above formula is actually simplified to:

[0095] ;

[0096] Among them, k is the category index corresponding to the true label.

[0097] For the entire first dataset label stream, the first tuning cost is the average of all single sample losses:

[0098] ;

[0099] Where N is the total number of samples in the label stream of the first dataset.

[0100] Assume that there are three labels in the label stream of the first data set, corresponding to the traffic conditions at three time points: "smooth", "mild congestion" and "moderate congestion". After processing in step S21b, the computer system obtains the first inference support coefficient vector corresponding to these three labels. For example, for the label "mild congestion", the support coefficient vector predicted by the model may be ([0.1, 0.7, 0.2]), indicating that the model believes that there is a 70% probability of mild congestion at this time point, while the true label is a one-hot vector ([0, 1, 0]). Based on these predicted values ​​and true labels, the computer system can calculate the cross entropy loss corresponding to each label, and average them to obtain the first tuning cost value. This value will be used in the subsequent algorithm parameter optimization process to minimize the loss and improve the inference accuracy of the model.

[0101] Step S21d: Optimizing the algorithm parameters of the original descriptor dimension compression algorithm according to the first tuning cost value, stopping the optimization when the first algorithm evaluation condition is reached, and obtaining the target descriptor dimension compression algorithm.

[0102] In step S21d, the computer system uses the first tuning cost value calculated in the previous step to optimize the algorithm parameters of the original descriptor dimension compression algorithm in order to reduce the loss value and improve the performance of the algorithm. This process is the core link of the target descriptor dimension compression algorithm training, which gradually approaches the optimal solution by iteratively adjusting the algorithm parameters. The following is a detailed explanation of this step, combined with a specific application scenario.

[0103] In the embodiment of the present application, the training of the target descriptor dimension compression algorithm is intended to extract a low-dimensional target sensor data set descriptor rich in key information from a high-dimensional original sensor data set descriptor. To achieve this goal, the computer system needs to continuously optimize the parameters of the algorithm so that the algorithm can output a more accurate target descriptor when processing a specific data set.

[0104] The specific method of optimizing algorithm parameters depends on the optimization algorithm used. In machine learning, commonly used optimization algorithms include gradient descent, stochastic gradient descent, Adam optimizer, etc. These algorithms calculate the gradient of the loss function with respect to the algorithm parameters and update the parameters in the opposite direction of the gradient, thereby gradually reducing the loss value.

[0105] Using gradient descent as an example, the computer system performs the following operations at each iteration:

[0106] Calculate the gradient: Based on the current algorithm parameters and the first optimization cost value (loss function value), calculate the gradient of the loss function with respect to the algorithm parameters. This gradient indicates the direction in which the loss function value decreases fastest.

[0107] Update parameters: Update the algorithm parameters in the opposite direction of the gradient, usually by subtracting the product of the gradient and the learning rate. The learning rate is a hyperparameter that controls the step size of the parameter update.

[0108] The optimization process will not go on indefinitely, it will stop when certain evaluation conditions are met. The evaluation conditions are usually related to factors such as the trend of the loss function value, the number of iterations, and the consumption of computing resources.

[0109] Change in loss function value: If the change in the loss function value is less than a preset threshold after multiple consecutive iterations, the algorithm is considered to have converged and the optimization can be stopped.

[0110] Iteration limit: To prevent the algorithm from falling into an infinite loop, a maximum number of iterations is usually set. When the number of iterations reaches this limit, the optimization will stop regardless of how the loss function value changes.

[0111] Computing resource consumption: In practical applications, computing resources are limited. If the computing resources consumed by the optimization process exceed the preset limit, the optimization needs to be stopped.

[0112] When the optimization process stops, the computer system obtains the optimized algorithm parameters, which define the specific implementation of the target descriptor dimension compression algorithm. At this point, the target descriptor dimension compression algorithm is ready for actual data processing tasks. It can efficiently extract the target sensor data set descriptor from the original sensor data set descriptor, providing strong support for subsequent intelligent traffic congestion warning analysis.

[0113] In step S21d, the computer system uses the gradient descent method to optimize the parameters of the original descriptor dimension compression algorithm. After multiple iterations, the loss function value gradually decreases and tends to be stable. When the change in the loss function value after multiple consecutive iterations is less than the preset threshold, the optimization process stops. At this point, the computer system obtains the optimized algorithm parameters and constructs the target descriptor dimension compression algorithm based on these parameters. The algorithm was then applied to real-time traffic data processing tasks, effectively supporting the operation of the intelligent traffic congestion warning system.

[0114] In a feasible implementation scheme, step S21b, according to the inference sensing data set descriptor corresponding to the first sequence, obtains the first inference support coefficient corresponding to the data set label at the first sequence in the first data set label stream, including:

[0115] Step S21b1: linearly map the inference sensor data set descriptor corresponding to the first sequence to obtain the original data set position descriptor corresponding to the first sequence; the original data set position descriptor includes description elements corresponding to each candidate sensor data set in the candidate sensor data set sequence;

[0116] Step S21b2: performing a standardization operation on the original data set position descriptor to obtain a first sequentially corresponding target data set position descriptor, wherein the target data set position descriptor includes first reasoning support coefficients corresponding to each candidate sensor data set in the candidate sensor data set sequence, and the candidate sensor data set sequence includes sensor data sets corresponding to each data set label in the first data set label stream;

[0117] Step S21b3: Determine, from the target dataset position descriptor, a first reasoning support coefficient corresponding to the dataset label at the first sequence in the first dataset label stream.

[0118] In the implementation of step S21b, the first reasoning support coefficient of the current first sequence data set label in the first data set label stream is derived from the first sequence corresponding reasoning sensor data set descriptor. This process not only involves mathematical transformations such as linear mapping and standardization, but also cleverly applies these transformations to the specific scenario of intelligent traffic information collection to achieve high-precision reasoning of traffic condition changes.

[0119] In step S21b1, the computer system linearly maps the first corresponding inference sensor data set descriptor to generate the original data set rank descriptor. The core of this step is to use linear transformation to map the eigenvalues ​​in the inference sensor data set descriptor to a new space, so that these eigenvalues ​​can more intuitively reflect the relative importance or rank relationship of each alternative sensor data set in the alternative sensor data set sequence in the new space. Linear mapping is a mapping that maintains the properties of vector addition and scalar multiplication operations. In the context of machine learning and data processing, linear mapping is usually implemented through matrix multiplication. Let the first corresponding inference sensor data set descriptor be vector ( ), where n is the number of features. The linear mapping can be represented as a matrix ( ) and the vector (x), that is, (Wx=y), where ( ) is the resulting vector after mapping.

[0120] In the embodiment of the present application, each row of the matrix (W) can be regarded as a weight vector, which defines the contribution of each feature in the original feature vector (x) to the corresponding element (i.e., description element) in the mapping result (y). These weights are learned through the algorithm training process, and they reflect the importance and mutual relationship of different traffic features in the reasoning process. For example, assume that the reasoning sensor data set descriptor is a vector containing three features (vehicle speed v, vehicle flow f, lane occupancy o). After linear mapping, an original data set position descriptor vector (y) is obtained, which contains the description elements of each alternative sensor data set in the alternative sensor data set sequence (i.e., the sensor data set corresponding to each data set label in the first data set label stream). These description elements may no longer be directly interpretable physical quantities, but comprehensive indicators obtained after linear combination for subsequent analysis and reasoning. In practical applications, due to the different dimensions of vehicle speed, vehicle flow, and lane occupancy, it is usually necessary to normalize these features first to ensure the effectiveness and rationality of linear mapping. Normalization can be achieved by methods such as minimum-maximum normalization and Z-score normalization.

[0121] In step S21b2, the computer system performs a standardization operation on the original data set position descriptor to generate a target data set position descriptor. The purpose of this step is to eliminate the dimensional differences and numerical range differences between different descriptive elements so that they can be compared and analyzed on the same scale. At the same time, the standardization operation also helps to improve the stability and accuracy of the subsequent reasoning process.

[0122] The standardization operation usually includes two steps: centering and scaling. Centering means subtracting the mean of each descriptive element to eliminate the offset; scaling means dividing the centered descriptive element by its standard deviation (or a fixed range) to reduce its numerical range. After standardization, the mean of the descriptive element becomes 0 and the standard deviation becomes 1 (or a specified fixed value).

[0123] In the embodiment of the present application, each element in the original data set position descriptor vector (y) can be standardized. Let yi be the i-th element in the vector y (i.e., the descriptor element of the i-th candidate sensor data set), μ and σ are the mean and standard deviation of the vector y (or a fixed scaling factor), respectively. Then the standardized element It can be expressed as:

[0124] ;

[0125] Or for normalization over a fixed range:

[0126] ;

[0127] Where min(y) and max(y) are the minimum and maximum values ​​in the vector (y), respectively.

[0128] After normalization, the target dataset position descriptor vector is obtained. , which contains the first reasoning support coefficients of each candidate sensor dataset in the candidate sensor dataset sequence. These coefficients are dimensionless and comparable on the same scale, and they reflect the reasoning correlation between the current first sequence dataset label and the candidate sensor dataset.

[0129] For example, continuing with the previous example, suppose the original dataset position descriptor vector (y) is centered and scaled to obtain the target dataset position descriptor vector .if The value of an element in (such as the first element) is 0.8, which means that the reasoning correlation between the first alternative sensor dataset (that is, the sensor dataset corresponding to the first dataset label in the first dataset label stream) and the current first sequential dataset label is high, and the model believes that the alternative sensor dataset has a greater contribution in reasoning about the current traffic conditions.

[0130] In step S21b3, the computer system determines the first reasoning support coefficient corresponding to the current first sequential dataset label in the first dataset label stream from the target dataset position descriptor. This step is relatively simple and direct, because the target dataset position descriptor has already expressed the reasoning correlation between each candidate sensor dataset in the candidate sensor dataset sequence and the current dataset label in a quantized form. In , directly find the element value at the index position corresponding to the current first-order dataset label, and the element value is the first reasoning support coefficient. Since the dataset label stream is arranged in chronological order, the position of the current first-order dataset label in the candidate sensor dataset sequence is known.

[0131] For example, suppose that the second dataset label in the first dataset label stream is currently being processed (i.e., the first order is 2), and the target dataset position descriptor vector has been generated. At this point, just look for The value of the element with index 2 in (assuming the index starts counting from 1) is the first inference support coefficient corresponding to the second dataset label. For example, if ( ), the first inference support coefficient of the second dataset label is 0.8.

[0132] In a feasible implementation scheme, the original descriptor dimension compression algorithm includes a feature extraction operator, a dimensionality reduction operator, a feature enhancement operator and a result output operator, the feature extraction operator is used to extract the original sensor data set descriptor, the dimensionality reduction operator is used for descriptor dimension compression, the feature enhancement operator is used to highlight the descriptor information, and the result output operator is used to output the first reasoning support coefficient. Then, step S21d, optimizes the algorithm parameters of the original descriptor dimension compression algorithm according to the first tuning cost value, stops the optimization when the first algorithm evaluation condition is reached, and obtains the target descriptor dimension compression algorithm, including: optimizes the algorithm parameters of the dimensionality reduction operator, the feature enhancement operator and the result output operator in the original descriptor dimension compression algorithm according to the first tuning cost value, stops the optimization when the first algorithm evaluation condition is reached, and obtains the target descriptor dimension compression algorithm.

[0133] In the implementation of step S21d, the computer system optimizes the key operator parameters in the original descriptor dimension compression algorithm to minimize the first tuning cost value, thereby obtaining a target descriptor dimension compression algorithm with better performance. First, it is necessary to clarify the basic structure of the original descriptor dimension compression algorithm, which consists of a feature extraction operator, a dimensionality reduction operator, a feature enhancement operator, and a result output operator. Each operator plays a different role in the algorithm:

[0134] Feature extraction operator: responsible for extracting key feature information from the original sensor data set to form the original sensor data set descriptor. These descriptors contain multi-dimensional features that characterize traffic conditions, such as vehicle speed, traffic volume, lane occupancy, etc. The feature extraction operator is, for example, an embedded network structure.

[0135] Dimensionality reduction operator: compresses the dimensions of the original sensor data set descriptor, removes redundant information, and retains key features. Dimensionality reduction operators reduce computational complexity by reducing the number of features while maintaining the original information of the data as much as possible. Commonly used dimensionality reduction techniques include PCA (principal component analysis) and LDA (linear discriminant analysis).

[0136] Feature enhancement operator: Perform information highlighting processing on the descriptor after dimensionality reduction, that is, enhance the features that contribute more to the reasoning process and suppress non-critical features. Feature enhancement operators can be implemented through attention mechanisms, feature weighting, etc. to improve the expressiveness of descriptors and the accuracy of reasoning.

[0137] Result output operator: Outputs the first inference support coefficient based on the enhanced descriptor, that is, the model's inference confidence or probability of the current traffic situation. The result output operator may be a simple linear layer or softmax layer, which is used to map the feature vector to the probability space.

[0138] In step S21d, the computer system uses the first tuning cost value as the optimization target to adjust the algorithm parameters of the dimensionality reduction operator, feature enhancement operator and result output operator in the original descriptor dimensionality compression algorithm. This process is usually implemented by optimization algorithms such as gradient descent method and Adam optimizer. For the dimensionality reduction operator, the optimization goal may be to find the optimal number of principal components, the optimal trade-off between the inter-class divergence and the intra-class divergence in LDA, etc. The adjustment of these parameters will directly affect the information retention and computational efficiency of the descriptor after dimensionality reduction. For example, in PCA, the computer system may try different numbers of principal components and observe the changes in the first tuning cost value to find the optimal value that minimizes the loss.

[0139] The optimization of the feature enhancement operator involves the adjustment of parameters such as attention weights and feature weighting coefficients. These parameters determine which features are more important in the reasoning process. By optimizing these parameters, the computer system can enhance the model's sensitivity to key features and improve the accuracy of reasoning. For example, in a feature enhancement operator based on the attention mechanism, the computer system may adjust the attention weights through the back-propagation algorithm so that the model can pay more attention to features that are useful for reasoning about the current traffic conditions.

[0140] The optimization of the result output operator mainly adjusts the weights and bias items of the linear layer or softmax layer to make the output first reasoning support coefficient more accurate. This step is similar to general machine learning classification or regression tasks. The computer system adjusts the parameters of the result output operator by minimizing functions such as cross entropy loss and mean square error loss.

[0141] The optimization process does not go on indefinitely, it stops when the first algorithm evaluation condition is met. The evaluation condition may include:

[0142] Convergence of loss value: If the change in the first tuning cost value after multiple consecutive iterations is less than a preset threshold, the algorithm is considered to have converged to a local optimal solution or near the global optimal solution, and the optimization can be stopped.

[0143] Iteration limit: To prevent the algorithm from falling into an infinite loop or overfitting the training data, a maximum number of iterations is usually set. When the number of iterations reaches this limit, the optimization will stop regardless of how the loss value changes.

[0144] Validation set performance: In practical applications, a portion of the data is usually reserved as a validation set to evaluate the performance of the model on unseen data. If the performance on the validation set begins to decline (i.e., overfitting occurs), the optimization should be stopped in time.

[0145] When any of the above evaluation conditions is met, the computer system stops the optimization process and uses the algorithm parameters obtained at this time as the final parameters of the target descriptor dimension compression algorithm. In this way, an optimized and better-performing descriptor dimension compression algorithm is obtained, which can more accurately extract key feature information from the original sensor data set and output a reliable first reasoning support coefficient, providing strong support for subsequent intelligent traffic congestion warning analysis.

[0146] In a feasible implementation scheme, the method further includes:

[0147] Step S21A: obtaining sensor data set contents corresponding to each candidate sensor data set in the candidate sensor data set sequence;

[0148] Step S21B: using the intelligent transportation big model, extracting descriptors from the sensor data set content, and obtaining original sensor data set descriptors corresponding to each candidate sensor data set;

[0149] Step S21C: obtaining an original sensor data set descriptor set according to the original sensor data set descriptors corresponding to the candidate sensor data sets.

[0150] In step S21A, the primary task of the computer system is to collect and organize traffic sensor data sets from different times and locations to form a sequence of candidate sensor data sets. These sensor data sets are usually collected by various sensors deployed on the road (such as geomagnetic sensors, microwave radars, infrared sensors, high-definition cameras, etc.), and contain rich traffic information.

[0151] The computer system pulls these sensor data sets regularly or in real time by integrating the data interfaces of various sensor devices, and sorts and organizes them according to time sequence and geographic location to form a sequence of candidate sensor data sets. Each candidate sensor data set corresponds to a record of traffic conditions at a specific time point and location. After obtaining the sequence of candidate sensor data sets, the computer system needs to use the smart traffic big model to analyze and process the contents of these data sets to extract key features that can characterize traffic conditions, namely the original sensor data set descriptors. The smart traffic big model is usually a large machine learning model or deep learning network that has been trained with a large amount of traffic data, and has powerful feature extraction and pattern recognition capabilities.

[0152] The smart transportation model may use a variety of advanced machine learning algorithms or neural network architectures, such as convolutional neural networks (CNN) for processing image data (such as road images taken by high-definition cameras), recurrent neural networks (RNN) or their variants (such as LSTM, GRU) for processing time series data (such as time series records of vehicle speed and traffic volume). In addition, advanced technologies such as attention mechanisms and graph neural networks can be combined to further improve the performance of the model.

[0153] During the feature extraction process, the model may learn key information such as the correlation between vehicle speed and traffic volume, the changing patterns of traffic conditions in different time periods, and encode this information into a series of high-dimensional vectors (i.e., descriptors of the original sensor data set). These descriptors not only contain the key feature information in the original data, but also extract deeper abstract features through the nonlinear transformation capabilities of the model.

[0154] In step S21B, the computer system has extracted the corresponding original sensor data set descriptor for each alternative sensor data set in the sequence of alternative sensor data sets. Next, the system needs to integrate these descriptors to form a set of original sensor data set descriptors. This set is a database containing a large number of descriptor vectors, each of which corresponds to a specific traffic condition record. In order to facilitate subsequent data processing and query operations, the computer system can further index and sort this sequence. For example, a unique identifier (such as a combination of timestamp and location information) can be assigned to each descriptor vector and stored in the database. At the same time, the descriptor vector can also be reduced in dimensionality (such as using dimensionality reduction techniques such as PCA) as needed to reduce storage space and computational complexity.

[0155] The resulting raw sensor data set descriptor set is a structured database that contains key feature information extracted from a large number of traffic sensor data sets. This information provides a rich data source for subsequent advanced applications such as data mining, pattern recognition, and traffic condition prediction.

[0156] Through the implementation of the above steps S21A to S21C, the computer system constructs a set of original sensor data set descriptors containing rich traffic feature information. This set not only provides solid basic data support for each module of the intelligent transportation system, but also reserves sufficient space for subsequent algorithm optimization and model upgrade.

[0157] In a feasible implementation scheme, step S220, according to the similarity measurement between the target road sensor dataset descriptors of the target road sensor dataset and the historical congestion sensor dataset, determines the approximate dataset sequence of the target road sensor dataset from the historical congestion sensor dataset sequence of the target road, including:

[0158] Step S221: according to the similarity measure between the target sensor data set descriptors of the target road sensor data set and the historical congestion sensor data set, based on the descending order of the similarity measure, determining a plurality of approximate data sets of the target road sensor data set from the sequence of the historical congestion sensor data sets of the target road;

[0159] Step S222: Arrange each approximate data set based on the congestion time sequence of the corresponding target road to obtain an approximate data set sequence of the target road sensor data set.

[0160] In step S221, the primary task of the computer system is to calculate the similarity between the target road sensor data set and the target sensor data set descriptors of the historical congestion sensor data set, and select multiple data sets that are most similar to the target road sensor data set from the historical congestion sensor data set sequence based on the result of the similarity measurement. Similarity measurement is a key step in evaluating the degree of similarity between two data sets. In the embodiment of the present application, commonly used similarity measurement methods include cosine similarity, Euclidean distance, Manhattan distance, etc. Among them, cosine similarity is particularly good at processing high-dimensional data because it can ignore the vector length (i.e., the absolute size of the eigenvalue) and only focus on the vector direction (i.e., the relative relationship between the features).

[0161] The computer system traverses the entire sequence of historical congestion sensor data sets, performs the above similarity calculation process on each historical congestion sensor data set, and sorts them in descending order according to the similarity value. Then, according to a preset threshold or a fixed number of principles, multiple historical congestion sensor data sets that are most similar to the target road sensor data set are selected from the sorted list as approximate data sets.

[0162] In step S222, the computer system arranges the multiple approximate data sets determined in step S221 according to their corresponding congestion time sequence of the target road in the historical records, thereby constructing an approximate data set sequence of the target road sensor data set.

[0163] Traffic congestion is constantly changing over time, so when constructing an approximate dataset sequence, the impact of the congestion time series must be fully considered. By arranging the approximate datasets in the order in which they appear in actual historical records, the dynamic evolution of traffic congestion can be more accurately reflected, providing strong data support for subsequent congestion warning analysis.

[0164] Assume that in step S221, the computer system selects the following three approximate data sets from the historical congestion sensor data set sequence (arranged in descending order of similarity): data set 1 (timestamp: 2023-03-30 14:50:00, similarity: 0.98); data set 2 (timestamp: 2023-03-30 15:05:00, similarity: 0.95); data set 3 (timestamp: 2023-03-30 14:40:00, similarity: 0.92)...

[0165] Next, the computer system arranges these approximate data sets in the congestion time sequence (i.e., time order) according to the timestamps corresponding to them, and obtains the approximate data set sequence of the target road sensor data set: [data set 3, data set 1, data set 2]. Although the similarity of data set 3 is slightly lower than that of data set 2, it is ranked first in the sequence because it appears before data set 1 on the timeline. This arrangement is more in line with the actual situation of traffic congestion evolving over time.

[0166] The constructed approximate dataset sequence not only contains historical congestion data that is highly similar to the current sensor dataset of the target road, but is also arranged in order according to the congestion time sequence. This allows subsequent analysis to more intuitively observe the entire process of traffic congestion from formation to dissipation, providing valuable data support for the formulation of effective traffic management strategies. At the same time, the sequence can also be used as input data for machine learning models to train more accurate congestion warning models, further improving the overall performance of the smart transportation system.

[0167] In a feasible implementation scheme, the method also includes a training process of the above-mentioned target machine learning algorithm, specifically including:

[0168] Step S1: Obtain a second data set label stream, and input the second data set label stream into an original machine learning algorithm; the second data set label stream is obtained based on a historical congestion sensor data set sequence of a second example road, and the original machine learning algorithm includes a priori sensor data descriptor set.

[0169] In the embodiment of the present application, each road carries a large amount of traffic data, which is collected in real time by various sensors deployed on the road. In order to train a target machine learning algorithm that can accurately predict traffic congestion, the computer system first needs to collect a sequence of historical congestion sensor data sets from a representative road (i.e., the second example road).

[0170] The historical congestion sensor data set sequence contains the traffic condition records of the road at different time points. Each record may contain multiple features such as vehicle speed, traffic volume, lane occupancy, vehicle type distribution, etc. In order to convert these raw data into the format required for algorithm training, the computer system performs a series of data processing steps, including data cleaning (removing noise, outliers, etc.), data normalization (converting data of different dimensions to the same scale) and feature extraction.

[0171] After completing the data processing, the computer system assigns a unique label to each record according to the time sequence. These labels not only contain the timestamp information, but also reflect the traffic congestion level at that time point (such as smooth, light congestion, moderate congestion, heavy congestion, etc.). In this way, the computer system constructs an ordered second data set label stream, in which each label corresponds to a specific time point and its traffic conditions in the second example road history record.

[0172] After preparing the second dataset label stream, the computer system passes it as input data to the original machine learning algorithm. The original machine learning algorithm is a pre-designed framework that contains multiple processing modules and parameter sets to learn the evolution law and feature representation of traffic congestion from the input data.

[0173] In particular, the original machine learning algorithm also includes a set of prior sensor data descriptors. This set is prepared before the algorithm is trained, and it contains key feature vectors extracted from a large amount of historical data. These feature vectors are carefully selected and compressed to efficiently represent key information under different traffic conditions. During the training process, the algorithm will use these prior descriptors to accelerate the learning process and improve prediction accuracy.

[0174] Within the algorithm, the set of prior sensor data descriptors already contains feature vector representations of similar roads at different congestion levels. For example, for the "moderate congestion" condition, the set may contain multiple feature vectors, which represent traffic conditions at different times and locations but with similar congestion levels. During the training process, the algorithm will use these prior descriptors to guide the reasoning process of the current dataset label, and gradually adjust the algorithm parameters through iterative optimization to minimize the prediction error. In this way, step S1 not only provides rich historical data support for the training process, but also accelerates the learning process of the algorithm by introducing prior knowledge. This lays a solid foundation for the subsequent construction of efficient and accurate target machine learning algorithms.

[0175] Step S2: using the original machine learning algorithm, in the set of prior sensor data descriptors, indexing to obtain the prior sensor data descriptors corresponding to each data set label before the second sequence in the second data set label stream, integrating the descriptor information of each prior sensor data descriptor obtained by the index, obtaining the inference sensor data set descriptor corresponding to the second sequence, and obtaining the second inference support coefficient corresponding to the data set label at the second sequence in the second data set label stream based on the inference sensor data set descriptor corresponding to the second sequence; the second sequence is determined in each data set label sequence of the second data set label stream, and the second inference support coefficient represents the inference correlation between the sensor data set corresponding to the second sequence and the forward data set sequence corresponding to the second sequence.

[0176] In step S2, the computer system first focuses on each dataset tag in the second dataset tag stream, especially those tags that are before the current processing sequence (i.e., the second sequence). For each such tag, the system performs an index operation in the prior sensor data descriptor set to find the corresponding prior sensor data descriptor.

[0177] The prior sensor data descriptor set is a database containing a large number of historical traffic condition feature vectors. These vectors are carefully selected and compressed to efficiently represent key information under different congestion levels and traffic conditions. The index operation is based on attributes such as timestamps and congestion levels in the label to ensure that the prior descriptor that best matches the current label is found.

[0178] After indexing all relevant prior sensor data descriptors, the computer system then performs descriptor information integration. This step aims to fuse the key information from multiple descriptors to form a more comprehensive and representative descriptor for the inference sensor data set.

[0179] Descriptor information integration may involve a variety of techniques, depending on the design of the original machine learning algorithm. For example, if the algorithm uses a recurrent neural network (RNN) or its variants (such as LSTM, GRU), information integration may be achieved by passing the hidden state of the network, which is spread out in time to capture dynamic changes in sequence data. Another common method is feature concatenation, which directly concatenates multiple descriptor vectors into a longer vector, which is then further processed by subsequent network layers.

[0180] After integrating the descriptor information, the computer system obtains the second-order corresponding inference sensor data set descriptor. This descriptor not only contains the traffic condition information before the current time point, but also makes a preliminary inference on the current traffic condition by integrating the key information from multiple prior descriptors. Next, the system calculates the second inference support coefficient based on the inference sensor data set descriptor. This coefficient represents the inference correlation between the second-order corresponding sensor data set (i.e., the traffic condition at the current time point) and the second-order corresponding forward data set sequence (i.e., the traffic condition sequence at all previous time points). In other words, it measures the probability or confidence of inferring the current traffic condition based on historical data.

[0181] The calculation of the second inference support coefficient may involve complex probability models or machine learning classifiers. For example, if the algorithm uses a softmax regression model as the output layer, the inference support coefficient can be obtained by mapping the inference sensor dataset descriptor to the probability space through the softmax function. The probability value of each category represents the possibility of the corresponding congestion level, and the probability value corresponding to the current label is the second inference support coefficient.

[0182] Finally, the system uses the softmax regression model to map the inference sensor dataset descriptor to the probability space and calculates the second inference support coefficient corresponding to the current label (i.e., the traffic situation at 8 a.m. on Monday). This coefficient may indicate that there is a 70% probability of being in a light congestion state at the current time point, thus providing an important reference for subsequent traffic management and congestion warning.

[0183] In this way, step S2 not only achieves the purpose of extracting key information from historical data, but also provides strong support for the prediction of current traffic conditions by calculating the inference sensor data set descriptor and the second inference support coefficient. This lays a solid foundation for building an efficient and accurate target machine learning algorithm.

[0184] Step S3: Obtain a second tuning cost value according to the second reasoning support coefficients corresponding to the dataset labels at each second sequence in the second dataset label stream.

[0185] In step S2, the computer system has calculated the corresponding second reasoning support coefficient for each data set label in the second data set label stream. These coefficients represent the probability or confidence of the model inferring the current traffic conditions based on historical data, and are the specific embodiment of the model prediction results.

[0186] To evaluate the performance of the model, the computer system needs to compare these inference support coefficients with the actual labels. The actual labels are the real records of traffic conditions at each time point in the label stream of the second dataset. They are directly provided by traffic management departments or sensor networks and have high accuracy and authority.

[0187] In order to quantify the difference between the model prediction results and the actual labels, the computer system defines a loss function. The loss function is an important indicator to measure the performance of the model. It is usually a non-negative real-valued function. The smaller the value, the better the prediction performance of the model.

[0188] In the embodiment of the present application, since traffic congestion level prediction is usually regarded as a multi-classification problem, the cross-entropy loss function can be used to calculate the second tuning cost value. The cross-entropy loss function can well handle the probability distribution differences in multi-classification problems and is one of the commonly used loss functions in classification tasks.

[0189] Taking a specific time point (such as 8 am on Monday) as an example, the actual label may indicate that the traffic condition at that time point is "moderate congestion", so its one-hot encoding is ([0, 0, 1, 0]) (assuming the category order is smooth, light congestion, moderate congestion, and severe congestion). The second inference support coefficient output by the model may be a probability distribution vector, such as ([0.1, 0.2, 0.6, 0.1]), indicating that the model believes that there is a 60% probability that the time point is in a state of moderate congestion.

[0190] According to the calculation formula of the cross entropy loss function, the computer system can calculate the loss value corresponding to the time point and add it to the total loss. Then, the system traverses all time points in the label stream of the second data set, repeats the above calculation process, and finally obtains the second tuning cost value (that is, the average loss value on the entire data set). By comparing the second tuning cost values ​​under different training rounds or different parameter settings, the computer system can evaluate the performance changes of the model and select the model with the best performance as the final target machine learning algorithm. This process not only helps to improve the prediction accuracy of the model, but also provides more reliable technical support for subsequent traffic management and congestion warning.

[0191] Step S4: Optimize the algorithm parameters of the original machine learning algorithm according to the second tuning cost value, stop the optimization when the second algorithm evaluation condition is reached, and obtain the target machine learning algorithm.

[0192] In step S3, the computer system has calculated the second tuning cost value, that is, the difference between the prediction result of the model under the current parameter settings and the actual label. In order to reduce this difference and improve the prediction accuracy of the model, the system needs to optimize the algorithm parameters of the original machine learning algorithm according to the second tuning cost value.

[0193] The optimization process usually involves gradient descent, Adam optimizer or other advanced optimization algorithms. These algorithms calculate the gradient of the loss function with respect to the algorithm parameters and update the parameters in the opposite direction of the gradient to gradually reduce the loss value. Specifically, for each parameter θ, its update rule can be expressed as:

[0194] ;

[0195] Among them, η is the learning rate, which controls the step size of parameter update; is the gradient of the loss function L with respect to the parameter θ.

[0196] In the embodiment of the present application, the original machine learning algorithm may be a complex neural network model, including multiple hidden layers, activation functions and weight parameters. The optimization algorithm needs to traverse all parameters in the model and update them according to their contribution to the loss function.

[0197] The optimization process will not go on indefinitely, it needs to stop when certain conditions are met. These conditions constitute the second algorithm evaluation criteria, which are used to determine whether the model has reached the expected performance level. Common evaluation conditions can refer to the content of S21d above in this application, and will not be repeated here.

[0198] During the training process, the computer system constructs a second data set label stream based on historical data, and calculates the second reasoning support coefficient at each time point through the original machine learning algorithm. Then, the system uses the cross entropy loss function to calculate the second tuning cost value, and optimizes the algorithm parameters of the model based on this value.

[0199] The optimization process is performed using the Adam optimizer, which can adaptively adjust the learning rate of each parameter to accelerate the convergence process. During the optimization process, the system regularly checks the changes in the loss value and the performance on the validation set to ensure that the model does not overfit.

[0200] After multiple rounds of iterations, when the loss value tends to be stable and the performance of the validation set no longer improves, the system stops the optimization process and uses the model obtained at this time as the target machine learning algorithm. The algorithm is then deployed in the smart transportation system to monitor and predict the congestion of urban roads in real time, providing timely and accurate decision-making basis for traffic management departments.

[0201] In a feasible implementation scheme, the original machine learning algorithm includes a feature extraction operator, a feature integration operator and a result output operator, the feature extraction operator is used to extract a priori sensor data descriptor, the feature integration operator is used to integrate descriptor information, and the result output operator is used to output a second reasoning support coefficient. Then, step S4, optimizing the algorithm parameters of the original machine learning algorithm according to the second tuning cost value, stopping the optimization when the second algorithm evaluation condition is reached, and obtaining the target machine learning algorithm, includes: optimizing the algorithm parameters of the feature integration operator and the result output operator in the original machine learning algorithm according to the second tuning cost value, stopping the optimization when the second algorithm evaluation condition is reached, and obtaining the target machine learning algorithm.

[0202] In the original machine learning algorithm, the feature extraction operator is responsible for extracting key feature information from the prior sensor data descriptor to provide basic data for subsequent processing. The feature integration operator is responsible for fusing and integrating these feature information to form a more comprehensive and representative inference sensor data set descriptor. Finally, the result output operator outputs the second inference support coefficient based on the integrated descriptor, that is, the model's prediction confidence for the current traffic conditions.

[0203] In step S4, the computer system does not directly optimize the parameters of the feature extraction operator (although this is necessary in some cases), but focuses on the feature integration operator and the result output operator. This is because the feature extraction operator is often closely related to the essential attributes of the data, and the adjustment of its parameters may require more domain knowledge and prior information. Feature integration and result output rely more on the structure and parameter settings of the model. By optimizing these parameters, the prediction performance of the model can be more effectively improved.

[0204] For the feature integration operator, the optimization process may involve adjusting the weight coefficient of the integration function, selecting a more appropriate integration strategy (such as weighted summation, attention mechanism, etc.), or changing the network structure of the integration layer. These adjustments are intended to enable the integrated descriptor to more accurately reflect the complex relationship between historical traffic conditions and current prediction targets.

[0205] For the result output operator, the optimization process focuses on the weight matrix and bias term of the output layer. In classification tasks, the output layer usually uses a softmax function to map the integrated descriptors to the probability space to output the predicted probability of each category. By adjusting the parameters of the output layer, the computer system can make the second inference support coefficient output by the model closer to the distribution of the true label.

[0206] In the optimization process, the second tuning cost value (i.e., the loss function value) plays a crucial guiding role. The computer system obtains the loss value by calculating the difference between the model prediction result and the actual label, and adjusts the parameters of the feature integration operator and the result output operator based on this value. The optimization algorithm (such as gradient descent method, Adam optimizer, etc.) will update the parameter value according to the gradient information of the loss function about the parameter, so as to gradually reduce the loss value and improve the prediction accuracy of the model.

[0207] In order to ensure the effectiveness and efficiency of the optimization process, the computer system sets a series of second algorithm evaluation conditions as stop criteria. These conditions may include the loss value converging below a preset threshold, the number of iterations reaching a maximum limit, or the performance of the validation set no longer improving. When any of these conditions is met, the optimization process will stop, and the algorithm parameters obtained at this time will be used as the final parameters of the target machine learning algorithm.

[0208] Then, the system enters the optimization process of step S4. At this stage, the computer system focuses on optimizing the parameters of the feature integration operator and the result output operator. The performance changes of the model are evaluated by continuously adjusting these parameter values ​​and calculating the second tuning cost value. After multiple rounds of iterations, when the loss value tends to be stable and meets the second algorithm evaluation conditions (such as the loss value is lower than the preset threshold), the optimization process stops and the target machine learning algorithm is obtained. The target machine learning algorithm finally obtained not only has the ability to extract key features from prior data, but also can predict future traffic congestion through efficient feature integration and accurate result output. This provides strong technical support for real-time monitoring, early warning and decision-making of intelligent transportation systems.

[0209] In a feasible implementation scheme, step S400 integrates descriptor information of each priori sensor data descriptor obtained by indexing to obtain a data sequence descriptor of the collaborative data set sequence, including:

[0210] Step S410: based on the data sets of the collaborative data set sequence, concatenate the priori sensor data descriptors obtained by the index to obtain a priori sensor data descriptor sequence;

[0211] Step S420: integrating descriptor information of the priori sensor data descriptor sequence to obtain a data set integration descriptor sequence, wherein the data set integration descriptor sequence includes data set integration descriptors corresponding to the number of data sets in the collaborative data set sequence;

[0212] Step S430: acquiring the dataset integration descriptor in the dataset sequence of the target road sensor dataset from the dataset integration descriptor sequence as the data sequence descriptor of the collaborative dataset sequence.

[0213] In step S410, the first task of the computer system is to sequentially splice the indexed prior sensor data descriptors according to the data sets of the collaborative data set sequence to form an ordered prior sensor data descriptor sequence. This process is the basis for the subsequent descriptor information integration, which ensures that each descriptor can be correctly processed in the context of its original timestamp or spatial position.

[0214] Assume that a historical congestion sensor data set sequence of a busy urban trunk road is being processed. The sequence contains traffic condition records at multiple time points (such as every hour), and each time point corresponds to a sensor data set. In steps S200 and S300, the computer system has screened out approximate data set sequences similar to the current sensor data set of the target road from the historical data, and indexed the prior sensor data descriptors corresponding to these approximate data sets through the target machine learning algorithm.

[0215] Now, in step S410, the computer system splices these prior sensor data descriptors one by one according to the time sequence of the collaborative data set sequence (i.e., the result of merging the current sensor data set and the approximate data set sequence of the target road). For example, if the collaborative data set sequence contains traffic condition records from 7 am to 9 pm, then the corresponding prior sensor data descriptors will also be sorted and spliced ​​according to this time period.

[0216] Assume that each prior sensor data descriptor is a vector containing multiple features, such as vehicle speed, traffic volume, lane occupancy, etc. Then, the concatenated prior sensor data descriptor sequence is a two-dimensional array or matrix, in which each row represents a descriptor vector at a time point. For example: Prior sensor data descriptor sequence = Among them, vi, fi, oi represent the characteristic values ​​of vehicle speed, traffic flow and lane occupancy rate at the i-th time point respectively.

[0217] In step S420, the computer system performs further information integration processing on the spliced ​​prior sensor data description subsequence. This step aims to extract key information from the sequence and fuse this information in some way to form a data set integration description subsequence that can more comprehensively reflect changes in traffic conditions. The specific method of information integration depends on the design of the target machine learning algorithm and the characteristics of the traffic data. In the embodiments of the present application, common information integration methods include but are not limited to feature weighted summation, principal component analysis (PCA), recurrent neural network (RNN) and its variants (such as LSTM, GRU), etc.

[0218] Weighted summation of features: Assign different weights to each descriptor vector in the sequence, and then perform weighted summation of the descriptors based on these weights. The weight assignment can be based on factors such as the similarity between descriptors and the time interval.

[0219] Principal Component Analysis (PCA): Maps high-dimensional descriptor vectors to low-dimensional space through linear transformation, while retaining the key information in the original data as much as possible. PCA can be used for dimensionality reduction to reduce the complexity of subsequent calculations.

[0220] Recurrent Neural Network (RNN): The RNN network structure is used to capture the temporal dependencies and dynamic changes in sequence data. The hidden state of RNN can be expanded in time to gradually accumulate and transmit key information in the sequence.

[0221] Assume that a recurrent neural network (RNN) is used as a method for information integration. In RNN, each time step receives an input (i.e., a priori sensor data descriptor vector) and updates its hidden state. The hidden state not only contains the information of the current time step, but also incorporates the information of all previous time steps. Therefore, the output processed by RNN can be regarded as an integrated description of the traffic conditions at the current and all previous time points.

[0222] Specifically, the computer system can use the prior sensor data descriptor sequence as the input sequence of the RNN and traverse each descriptor vector in the sequence. For each descriptor vector, the RNN will calculate a corresponding hidden state vector and use it as part of the dataset integration descriptor at that time point. After traversing the entire sequence, a dataset integration descriptor sequence is obtained, in which each descriptor is an integrated representation of the traffic conditions at all previous time points.

[0223] It is worth noting that the output layer of the RNN may need further processing to obtain the final dataset integration descriptor. For example, a fully connected layer can be used to map the hidden state vector of the RNN into a fixed-dimensional vector space to form a standardized dataset integration descriptor.

[0224] Although the calculation process of RNN is relatively complex and difficult to express with a simple formula, the embodiment of the present application provides a general description thereof. Assume that the hidden state update function of RNN is:

[0225] ;

[0226] where h t is the hidden state vector at the current time step, σ is the activation function (such as tanh or ReLU), and W h and W x is the weight matrix, b h is the bias term, x t is the input vector of the current time step (i.e., the prior sensor data descriptor vector). Then, by traversing the entire input sequence and updating the hidden state, a series of data set integration descriptors (i.e., hidden state vectors) can be obtained.

[0227] In step S430, the computer system extracts the data set integration descriptor on the target road sensor data set sequence from the data set integration descriptor sequence as the data sequence descriptor of the collaborative data set sequence. This process is a summary and refinement of all the previous steps, which ensures that the final data sequence descriptor can accurately reflect the current and previous traffic conditions of the target road.

[0228] Since the collaborative dataset sequence is arranged in chronological order, the dataset integration descriptor in the target dataset sequence can be directly found through the index. For example, if the current sensing dataset of the target road corresponds to the traffic condition record at 8 o'clock in the morning, and the dataset integration descriptor sequence obtained in step S420 is also arranged in chronological order, then the descriptor at the 8th position in the sequence (assuming that the index starts from 1) can be directly extracted as the data sequence descriptor.

[0229] This data sequence descriptor not only contains the traffic condition information of the target road at the current time point, but also integrates the key information of all previous time points through the previous descriptor information integration process. Therefore, it can be used as important input data for subsequent congestion warning, road condition analysis and other applications.

[0230] In a feasible implementation scheme, the method further includes:

[0231] Step S500: obtaining a sensor data set descriptor of a target road sensor data set and a road attribute descriptor of the target road;

[0232] Step S600: the data sequence descriptor, the sensor data set descriptor of the target road sensor data set, and the road attribute descriptor of the target road are input into the target reasoning algorithm to obtain the confidence level of the target road sensor data set in representing the congestion warning of the target road.

[0233] The introduction of steps S500 and S600 provides the system with a more comprehensive and sophisticated congestion warning capability. These two steps not only consider the information of the sensor data set itself, but also incorporate the road attribute descriptor of the target road and the data sequence descriptor generated by the collaborative data set sequence, thereby realizing a multi-dimensional analysis and comprehensive evaluation of the traffic conditions.

[0234] In step S500, the primary task of the computer system is to obtain the sensor data set descriptor of the target road sensor data set and the road attribute descriptor of the target road. These two pieces of information are crucial for the subsequent congestion warning analysis, as they represent the direct observation data of the traffic conditions and the inherent attributes of the road itself, respectively. The sensor data set descriptor is an abstract representation of the sensor data set feature vector, which contains key information extracted from the original sensor data. In an embodiment of the present application, the sensor data set may be collected in real time by various sensors deployed on the road (such as geomagnetic sensors, microwave radars, high-definition cameras, etc.), and includes various features such as vehicle speed, traffic volume, lane occupancy, and vehicle type distribution.

[0235] The computer system processes the original sensor data set through feature extraction algorithms (such as PCA, LDA and other dimensionality reduction techniques or feature extraction layers in deep learning networks) to generate sensor data set descriptors. These descriptors not only reduce the dimensionality and complexity of the data, but also enhance the expressiveness and discrimination of the data.

[0236] The road attribute descriptor reflects the inherent attributes of the target road itself, such as road type (highway, urban main road, secondary road, etc.), number of lanes, road width, traffic light settings, etc. This information is of great significance for understanding the formation mechanism and evolution law of traffic congestion.

[0237] The computer system can obtain the road attribute descriptor of the target road by accessing the database of the traffic management department or the geographic information system (GIS). These attributes usually exist in the form of structured data, such as key-value pairs or table data. For example, the road attribute descriptor of the target road may contain the following information:

[0238] Road type: urban trunk road; number of lanes: six lanes in both directions; road width: 30 meters; traffic light settings: yes (one every 500 meters on average)... This information is integrated into a road attribute descriptor vector or dictionary for subsequent analysis and reasoning.

[0239] In step S600, the computer system passes the data sequence descriptor obtained in step S400, the sensor data set descriptor of the target road sensor data set obtained in step S500, and the road attribute descriptor of the target road into the target reasoning algorithm. The target reasoning algorithm is a reasoning model obtained through labeled training, which can comprehensively evaluate the congestion status of the target road based on the input descriptor information and output the corresponding congestion warning confidence.

[0240] In the training phase, the target inference algorithm performs supervised learning based on training sample pairs with training labels. Each training sample pair includes the sensor dataset descriptor of the training sensor dataset, the road attribute descriptor of the example road, the fusion feature between the training sensor dataset and the example road, and the corresponding training label.

[0241] The fusion feature is obtained by combining the sensor dataset descriptor and the road attribute descriptor in some way, and it aims to capture the interaction and common influence between the two. Fusion methods may include feature concatenation, weighted summation, feature intersection, etc. For example, the sensor dataset descriptor and the road attribute descriptor can be directly concatenated into a longer vector as a fusion feature. The training label is used to indicate whether there is a match or corresponding representation between the training sensor dataset and the example road. If the training sensor dataset is collected when the example road is congested, the training label is a positive label; otherwise, it is a negative label.

[0242] By learning from a large number of training sample pairs, the target reasoning algorithm can learn the complex relationship between the sensor data set descriptors, road attribute descriptors, and their fusion features and congestion conditions, thereby having the ability to accurately reason about unknown data.

[0243] In the inference stage, the computer system passes the sensor data set descriptor of the target road sensor data set collected in real time, the road attribute descriptor of the target road, and the data sequence descriptor generated by the collaborative data set sequence as input to the target inference algorithm. The algorithm processes and analyzes the input data through a series of complex calculation processes (such as the forward propagation of the neural network), and finally outputs a value representing the confidence of the congestion warning. This confidence value reflects the credibility of the target road sensor data set in representing the current or future congestion condition of the target road. If the confidence is greater than the preset threshold (such as 0.8), the computer system can assume that the target road is about to or is experiencing congestion, and send an early warning signal to the traffic management department. On the contrary, if the confidence is lower than the threshold, the system may believe that the current traffic condition is relatively smooth or the risk of congestion is low.

[0244] At a specific time point (such as 8 am on a weekday), the computer system first collects the real-time sensor data set of the target road (such as a busy urban main road) and generates a sensor data set descriptor through a feature extraction algorithm. At the same time, the system obtains the road attribute descriptor of the target road from the database or GIS of the traffic management department. Then, the system uses the target machine learning algorithm to analyze the historical congestion sensor data set sequence to generate a data sequence descriptor of the collaborative data set sequence. This descriptor not only contains the traffic condition information at the current time point, but also integrates the key information of the previous time points to reflect the dynamic changes of the traffic conditions. Finally, the system passes the sensor data set descriptor, road attribute descriptor and data sequence descriptor into the target reasoning algorithm. After a series of complex calculations, the algorithm outputs a value (such as 0.9) representing the congestion warning confidence. Since this confidence is greater than the preset threshold (such as 0.8), the system determines that the target road is about to or is experiencing congestion, and immediately sends a warning signal to the traffic management department so that timely countermeasures can be taken. Through such a process, the intelligent transportation system not only realizes the rapid processing and accurate analysis of real-time traffic data, but also improves the accuracy and reliability of congestion warning by introducing road attribute descriptors and data sequence descriptors of collaborative data set sequences. This provides strong technical support and decision-making basis for urban traffic management and promotes the smooth operation and sustainable development of urban traffic.

[0245] The method provided in this embodiment can also be applied to other road traffic data processing after adaptive adjustment. For example, when conducting safety management and control of passenger and freight transport vehicles, the driver behavior data and vehicle travel data obtained by the on-board satellite positioning active safety terminal can be analyzed and processed to judge the driver's behavior habits, improve the driver's safe driving ability, and enhance road safety.

[0246] Specifically, it can be done by collecting on-board sensor data: on-board DSN sensor data (driver driving behavior data such as fatigue driving, distracted driving, making phone calls, hands off the steering wheel, smoking, not looking forward for a long time, etc. as analyzed by video), on-board WeChat positioning data (vehicles speeding and not following the prescribed route as determined by BD satellite positioning), on-board ADS sensor data (vehicles too close to each other, not changing lanes as required, etc.), vehicle speeding determined by satellite positioning), tire pressure sensors, etc.

[0247] Through the Beidou satellite positioning system, combined with machine learning or algorithms, the driver's driving behavior can be analyzed, the vehicle's safe driving capabilities can be improved, and the safety of vehicles and roads can be improved. Figure 2 A hardware entity diagram of a computer system provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the hardware entity of the computer system 1000 includes: a processor 1001 and a memory 1002, wherein the memory 1002 stores a computer program that can be run on the processor 1001, and the processor 1001 implements the steps in the method of any of the above embodiments when executing the program.

[0248] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A method for collecting intelligent traffic information based on big data, characterized in that: The method comprises: The traffic sensor network deployed on the target road collects the target road sensor data set and obtains the historical congestion sensor data set sequence of the target road; Determining an approximate data set sequence of the target road sensor data set from a historical congestion sensor data set sequence of the target road; A collaborative data set sequence is obtained according to the approximate data set sequence and the target road sensor data set, and the collaborative data set sequence is passed to a target machine learning algorithm; the target machine learning algorithm includes a priori sensor data descriptor set, and the target machine learning algorithm is tuned according to a historical congestion sensor data set sequence of an example road; The target machine learning algorithm is adopted to index the prior sensor data descriptor set corresponding to each sensor data set in the collaborative data set sequence, and descriptor information of each prior sensor data descriptor obtained by the index is integrated to obtain the data sequence descriptor of the collaborative data set sequence. The data sequence descriptor is used to determine the confidence level of the target road sensor data set in representing the target road congestion warning.

2. The method according to claim 1, characterized in that The step of determining an approximate data set sequence of the target road sensor data set from the historical congestion sensor data set sequence of the target road comprises: For the target road sensor data set and the historical congestion sensor data set in the historical congestion sensor data set sequence of the target road, extract a target sensor data set descriptor of the sensor data set; wherein the target sensor data set descriptor is obtained by performing a descriptor dimension compression operation on the original sensor data set descriptor of the sensor data set; According to the similarity measure between the target road sensor dataset descriptors and the historical congestion sensor dataset descriptors, an approximate dataset sequence of the target road sensor dataset is determined from the historical congestion sensor dataset sequence of the target road.

3. The method according to claim 2, characterized in that The step of extracting a target sensor data set descriptor of the sensor data set from the target road sensor data set and the historical congestion sensor data set in the sequence of historical congestion sensor data sets of the target road comprises: A target data set label stream is obtained according to the data set label formation of the current sensor data set, and the target data set label stream is input into a target descriptor dimension compression algorithm; the target data set is the target road sensor data set or a historical congestion sensor data set in a historical congestion sensor data set sequence of the target road, and the target descriptor dimension compression algorithm includes an original sensor data set descriptor set; The target descriptor dimension compression algorithm is adopted to obtain, in the original sensor data set descriptor set, the original sensor data set descriptor corresponding to the data set label of the current sensor data set by indexing, and the descriptor information is highlighted and the descriptor dimension is compressed on the indexed original sensor data set descriptor to obtain the target sensor data set descriptor of the current sensor data set.

4. The method according to claim 3, characterized in that The method further comprises: Acquire a first data set label stream, and input the first data set label stream into an original descriptor dimension compression algorithm; the first data set label stream is obtained according to a historical congestion sensor data set sequence of a first example road, and the original descriptor dimension compression algorithm includes the original sensor data set descriptor set; The original descriptor dimension compression algorithm is adopted, and in the original sensor data set descriptor set, the original sensor data set descriptor corresponding to each data set label before the first sequence in the first data set label stream is indexed, and the descriptor information of each original sensor data set descriptor obtained by the index is highlighted and the descriptor dimension is compressed to obtain the inference sensor data set descriptor corresponding to the first sequence, and the first inference support coefficient corresponding to the data set label on the first sequence in the first data set label stream is obtained according to the inference sensor data set descriptor corresponding to the first sequence; the first sequence is determined in each data set label sequence of the first data set label stream, and the first inference support coefficient represents the inference correlation between the sensor data set corresponding to the first sequence and the forward data set sequence of the first sequence; Obtaining a first tuning cost value according to first reasoning support coefficients respectively corresponding to the dataset labels at each first sequence in the first dataset label stream; The algorithm parameters of the original descriptor dimension compression algorithm are optimized according to the first tuning cost value, and the optimization is stopped when the first algorithm evaluation condition is reached to obtain the target descriptor dimension compression algorithm.

5. The method according to claim 4, characterized in that The obtaining, according to the inference sensing data set descriptor corresponding to the first sequence, a first inference support coefficient corresponding to the data set label at the first sequence in the first data set label stream includes: Linearly mapping the inference sensor data set descriptor corresponding to the first sequence to obtain the original data set position descriptor corresponding to the first sequence; the original data set position descriptor includes description elements corresponding to each candidate sensor data set in the candidate sensor data set sequence; Performing a normalization operation on the original data set position descriptor to obtain a target data set position descriptor corresponding to the first sequence, wherein the target data set position descriptor includes a first reasoning support coefficient corresponding to each candidate sensor data set in the candidate sensor data set sequence, and the candidate sensor data set sequence includes a sensor data set corresponding to each data set label in the first data set label stream; Determining, from the target data set position descriptor, a first reasoning support coefficient corresponding to the data set label at the first sequence in the first data set label stream; The original descriptor dimension compression algorithm includes a feature extraction operator, a dimension reduction operator, a feature enhancement operator and a result output operator, wherein the feature extraction operator is used to extract the original sensor data set descriptor, the dimension reduction operator is used to compress the descriptor dimension, the feature enhancement operator is used to highlight the descriptor information, and the result output operator is used to output the first reasoning support coefficient; The step of optimizing the algorithm parameters of the original descriptor dimension compression algorithm according to the first tuning cost value, stopping the optimization when the first algorithm evaluation condition is reached, and obtaining the target descriptor dimension compression algorithm includes: According to the first tuning cost value, the algorithm parameters of the dimensionality reduction operator, the feature enhancement operator and the result output operator in the original descriptor dimension compression algorithm are optimized, and the optimization is stopped when the first algorithm evaluation condition is reached to obtain the target descriptor dimension compression algorithm.

6. The method according to claim 3, characterized in that The method further comprises: Obtaining sensor data set contents corresponding to each candidate sensor data set in the candidate sensor data set sequence; Using the intelligent transportation big model, extracting descriptors from the sensor data set content, and obtaining original sensor data set descriptors corresponding to each of the candidate sensor data sets; Obtaining the original sensor data set descriptor set according to the original sensor data set descriptors respectively corresponding to the candidate sensor data sets; Determining an approximate data set sequence of the target road sensor data set from the historical congestion sensor data set sequence of the target road according to the similarity measurement between the target sensor data set descriptors of the target road sensor data set and the historical congestion sensor data set comprises: According to a similarity measure between the target sensor dataset descriptors of the target road sensor dataset and the historical congestion sensor dataset, respectively, determining a plurality of approximate datasets of the target road sensor dataset from a sequence of historical congestion sensor datasets of the target road based on a descending order of the similarity measure; The approximate data sets are arranged based on the congestion time sequence corresponding to the target road to obtain an approximate data set sequence of the target road sensor data set.

7. The method according to claim 1, characterized in that The method further comprises: Acquire a second data set label stream, and input the second data set label stream into an original machine learning algorithm; the second data set label stream is obtained according to a historical congestion sensor data set sequence of a second example road, and the original machine learning algorithm includes the prior sensor data descriptor set; Using the original machine learning algorithm, in the prior sensor data descriptor set, indexing to obtain the prior sensor data descriptor corresponding to each data set label before the second sequence in the second data set label stream, integrating descriptor information of each prior sensor data descriptor obtained by the index, obtaining the inference sensor data set descriptor corresponding to the second sequence, and obtaining the second inference support coefficient corresponding to the data set label at the second sequence in the second data set label stream according to the inference sensor data set descriptor corresponding to the second sequence; The second sequence is determined in each data set label sequence in the second data set label stream, and the second reasoning support coefficient represents the reasoning correlation between the sensing data set corresponding to the second sequence and the forward data set sequence corresponding to the second sequence; Obtaining a second tuning cost value according to second reasoning support coefficients corresponding to the data set labels at each second sequence in the second data set label stream; The algorithm parameters of the original machine learning algorithm are optimized according to the second tuning cost value, and the optimization is stopped when the second algorithm evaluation condition is reached to obtain the target machine learning algorithm.

8. The method according to claim 7, characterized in that The original machine learning algorithm includes a feature extraction operator, a feature integration operator and a result output operator, wherein the feature extraction operator is used to extract a priori sensor data descriptors, the feature integration operator is used to integrate descriptor information, and the result output operator is used to output a second reasoning support coefficient; the algorithm parameters of the original machine learning algorithm are optimized according to the second tuning cost value, and the optimization is stopped when the second algorithm evaluation condition is reached to obtain the target machine learning algorithm, including: The algorithm parameters of the feature integration operator and the result output operator in the original machine learning algorithm are optimized according to the second tuning cost value, and the optimization is stopped when the second algorithm evaluation condition is reached to obtain the target machine learning algorithm.

9. The method according to claim 1, characterized in that: The step of integrating descriptor information of each priori sensor data descriptor obtained by indexing to obtain a data sequence descriptor of the collaborative data set sequence includes: Based on the data set sequence of the collaborative data set sequence, concatenating the priori sensor data descriptors obtained by the index to obtain a priori sensor data descriptor sequence; Integrating descriptor information of the priori sensor data descriptor sequence to obtain a data set integration descriptor sequence, wherein the data set integration descriptor sequence includes data set integration descriptors corresponding to the number of data sets in the collaborative data set sequence; From the dataset integration descriptor sequence, obtaining a dataset integration descriptor in the dataset sequence of the target road sensor dataset as a data sequence descriptor of the collaborative dataset sequence; The method further comprises: Acquire a sensor data set descriptor of the target road sensor data set and a road attribute descriptor of the target road; The data sequence descriptor, the sensor data set descriptor of the target road sensor data set, and the road attribute descriptor of the target road are input into the target reasoning algorithm to obtain the confidence level of the target road sensor data set in representing the congestion warning of the target road.

10. A computer system comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, wherein: When the processor executes the program, the steps in the method according to any one of claims 1 to 9 are implemented.

Images