Decision support information method and system based on data fusion analysis and electronic equipment

Abstract

The invention relates to the field of decision support information, and particularly discloses a decision support information method and system based on data fusion analysis and electronic equipment. The method comprises the following steps: firstly, acquiring coordinate text data of each camera deployed on a road, a plurality of road monitoring videos acquired by the plurality of cameras deployed on the road and traffic flow of each road section in a plurality of preset time periods, then, performing feature extraction and correlation analysis on the coordinate text data, the plurality of road monitoring videos and the traffic flow in the plurality of preset time periods by using a deep learning technology, and finally, obtaining a decoding value through a decoder, the traffic flow prediction value of the road section to be predicted at the next moment is predicted, more comprehensive and accurate information is provided for traffic management and decision making, and the traffic transportation efficiency and the urban traffic quality are improved.

Description

Technical Field

[0001] The present application relates to the field of decision support information, and more specifically, to a decision support information method, system and electronic equipment based on data fusion analysis. Background Art

[0002] Urban road network traffic condition analysis refers to the evaluation of traffic flow, congestion and traffic operation efficiency of the urban internal road network by collecting and analyzing traffic data. The purpose of this analysis method is to deal with traffic congestion problems, improve traffic operation efficiency, and thus improve the travel experience of urban residents.

[0003] At present, the analysis of traffic congestion mainly relies on monitoring equipment installed at intersections and key sections where traffic is relatively concentrated. However, due to the angle and line of sight problems of single-view monitoring equipment, the monitoring results may be biased. In addition, if the data collected by these monitoring devices are analyzed manually, it will require a lot of manpower costs and will also be affected by human factors.

[0004] Therefore, a decision support information method, system and electronic device based on data fusion analysis are desired. Summary of the invention

[0005] In order to solve the above technical problems, this application is proposed. The embodiments of this application provide a decision support information method, system and electronic device based on data fusion analysis, which first obtains the coordinate text data of each camera deployed on the road, multiple road monitoring videos collected by multiple cameras deployed on the road and the traffic flow of each road section in multiple predetermined time periods, and then uses deep learning technology to perform feature extraction and correlation analysis on the three, and finally obtains the decoded value through the decoder to predict the traffic flow prediction value of the road section to be predicted at the next moment, thereby providing more comprehensive and accurate information for traffic management and decision making, thereby helping to improve transportation efficiency and the quality of urban transportation.

[0006] According to one aspect of the present application, a decision support information method based on data fusion analysis is provided, which includes:

[0007] Obtaining coordinate text data of each camera deployed on the road, multiple road monitoring videos collected by multiple cameras deployed on the road, and traffic flow of each road section in multiple predetermined time periods, wherein the coordinate text data includes the X-axis coordinate, Y-axis coordinate, and Z-axis coordinate of each camera;

[0008] Extracting a road camera feature matrix and a traffic flow global correlation feature matrix from the coordinate text data of each camera deployed on the road, a plurality of road monitoring videos collected by the plurality of cameras deployed on the road, and the traffic flow of each road section in a plurality of predetermined time periods;

[0009] Based on the road camera feature matrix and the vehicle flow global correlation feature matrix, the traffic flow prediction value of the road section to be predicted at the next moment is predicted.

[0010] According to another aspect of the present application, a decision support information system based on data fusion analysis is provided, comprising:

[0011] A traffic flow data acquisition module, used to acquire coordinate text data of each camera deployed on the road, multiple road monitoring videos collected by multiple cameras deployed on the road, and traffic flow of each road section in multiple predetermined time periods, wherein the coordinate text data includes the X-axis coordinate, Y-axis coordinate, and Z-axis coordinate of each camera;

[0012] A traffic flow data feature extraction module, used to extract a road camera feature matrix and a traffic flow global correlation feature matrix from the coordinate text data of each camera deployed on the road, the multiple road monitoring videos collected by the multiple cameras deployed on the road, and the traffic flow of each road section in multiple predetermined time periods;

[0013] The traffic flow prediction generation module is used to predict the traffic flow prediction value of the road section to be predicted at the next moment based on the road camera feature matrix and the traffic flow global correlation feature matrix.

[0014] Compared with the prior art, the present application provides a decision support information method, system and electronic device based on data fusion analysis, which first obtains the coordinate text data of each camera deployed on the road, multiple road monitoring videos collected by multiple cameras deployed on the road and the traffic flow of each road section in multiple predetermined time periods, and then uses deep learning technology to perform feature extraction and correlation analysis on the three, and finally obtains the decoded value through the decoder to predict the traffic flow prediction value of the road section to be predicted at the next moment, thereby providing more comprehensive and accurate information for traffic management and decision making, thereby helping to improve transportation efficiency and the quality of urban transportation. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0016] Figure 1 The present invention is a flowchart of a decision support information method based on data fusion analysis according to an embodiment of the present application.

[0017] Figure 2 The present invention is a flowchart for extracting features from the coordinate text data of each camera deployed on the road in the decision support information method based on data fusion analysis according to an embodiment of the present application.

[0018] Figure 3 The present invention is a flowchart for predicting the traffic flow prediction value of the road section to be predicted at the next moment based on the road camera feature matrix and the vehicle flow global correlation feature matrix in the decision support information method based on data fusion analysis according to an embodiment of the present application.

[0019] Figure 4 It is a block diagram of a decision support information system based on data fusion analysis according to an embodiment of the present application.

[0020] Figure 5 is a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0022] Figure 1 FIG. 1 is a flow chart of a decision support information method based on data fusion analysis according to an embodiment of the present application. Figure 1 As shown, according to the decision support information method based on data fusion analysis of the embodiment of the present application, it includes: S110, obtaining the coordinate text data of each camera deployed on the road, a plurality of road monitoring videos collected by a plurality of cameras deployed on the road, and the traffic flow of each road section in a plurality of predetermined time periods, wherein the coordinate text data includes the X-axis coordinate, Y-axis coordinate and Z-axis coordinate of each camera; S120, extracting the road camera feature matrix and the traffic flow global correlation feature matrix from the coordinate text data of each camera deployed on the road, the plurality of road monitoring videos collected by a plurality of cameras deployed on the road, and the traffic flow of each road section in a plurality of predetermined time periods; S130, predicting the traffic flow prediction value of the road section to be predicted at the next moment based on the road camera feature matrix and the traffic flow global correlation feature matrix.

[0023] In the above-mentioned decision support information method based on data fusion analysis, the step S110 obtains the coordinate text data of each camera deployed on the road, multiple road monitoring videos collected by multiple cameras deployed on the road, and the traffic flow of each road section in multiple predetermined time periods, wherein the coordinate text data includes the X-axis coordinate, Y-axis coordinate and Z-axis coordinate of each camera. It should be understood that considering that only a traffic monitoring device with a single perspective will cause the monitoring result to deviate due to the problem of angle and line of sight, therefore, in the technical solution of the present application, the traffic flow is predicted and decision support is provided through the coordinate text data of each camera deployed on the road, the road monitoring video and the multi-angle data of the traffic flow data. These data can provide a comprehensive understanding of the traffic conditions, thereby supporting applications such as traffic management, road condition prediction, and traffic planning. Specifically, obtaining the coordinate text data of the camera can determine the position of each camera in the road network. The X-axis, Y-axis and Z-axis coordinates in the coordinate text data provide the position information of the camera in three-dimensional space. These coordinate data can be used to determine the specific position of the camera, and then model the road network and locate the vehicle position. Obtaining the road monitoring video can provide real-time road condition information. By analyzing video data, traffic flow, vehicle speed, vehicle density, and road congestion can be detected. Obtaining traffic flow data for each road section at multiple predetermined time periods can provide traffic data for analyzing traffic cyclical changes, peak hours, congestion conditions, etc. This data can help predict future traffic flows and support the formulation of traffic management decisions.

[0024] In the above-mentioned decision support information method based on data fusion analysis, the step S120 extracts the road camera feature matrix and the traffic flow global correlation feature matrix from the coordinate text data of each camera deployed on the road, the multiple road monitoring videos collected by the multiple cameras deployed on the road, and the traffic flow of each road section in multiple predetermined time periods. It should be understood that the road camera feature matrix can extract features such as the field of view, shooting angle, and resolution of the road camera. These features can be used to determine the coverage and monitoring capabilities of the camera, thereby affecting the observation and analysis of traffic flow. The traffic flow global correlation feature matrix can extract the correlation between traffic flows between different sections. These features may include information such as vehicle speed, density, and flow. By extracting the traffic flow global correlation feature matrix, the traffic flow change trend and mutual influence relationship between different sections can be analyzed, so as to better understand the overall situation and congestion of the traffic network.

[0025] In a specific embodiment of the present application, a road camera feature matrix and a traffic flow global correlation feature matrix are extracted from the coordinate text data of each camera deployed on the road, the multiple road monitoring videos collected by the multiple cameras deployed on the road, and the traffic flow of each road section in multiple predetermined time periods, including: feature extraction of the coordinate text data of each camera deployed on the road to obtain a coordinate global correlation feature matrix; feature extraction of the multiple road monitoring videos collected by the multiple cameras deployed on the road to obtain multiple monitoring key frame feature matrices; point multiplication of the coordinate global correlation feature matrix and the multiple monitoring key frame feature matrices to obtain the road camera feature matrix; feature extraction of the traffic flow of each road section in multiple predetermined time periods to obtain the traffic flow global correlation feature matrix. It should be understood that, considering that the topological structure of the road network can be revealed by analyzing the global correlation feature matrix of the camera coordinates, including the connection relationship of the roads, the location of the intersection, the distance between the roads, etc. This is very important for traffic planning and traffic management, and can help decision makers understand the overall situation of the road network and optimize traffic routes and traffic flow control. At the same time, through the coordinate global correlation feature matrix, the distance between cameras, angle difference and other indicators can be calculated to evaluate the coverage and monitoring capabilities of the cameras. This is of great significance for selecting the best camera position, determining the monitoring area and monitoring effect. In addition, through the coordinate global correlation feature matrix, the degree of correlation between the sections monitored by different cameras can be analyzed. This helps to understand the propagation and distribution of traffic flow in the road network, identify bottleneck sections and congested areas, and provide more accurate traffic flow prediction and congestion management. In this way, the spatial relationship between cameras can be better understood and described, so that traffic flow can be analyzed more accurately and comprehensively. Therefore, the coordinate text data of each camera deployed on the road is feature extracted to obtain the coordinate global correlation feature matrix, which is used to analyze the topological structure of the road network and the degree of correlation between cameras. Further, considering that the surveillance video usually contains a large number of continuous frame images, directly processing the entire video may result in a huge amount of calculation. In the technical solution of this application, by extracting key frames, the video content is summarized and only representative frame images are retained. This can greatly reduce the amount of data processed and improve processing efficiency.

[0026] Furthermore, the coordinate global correlation feature matrix provides the spatial position relationship between cameras, while multiple monitoring key frame feature matrices provide the features of the video content. By dot-multiplying them by position, the spatial information and video content features can be integrated to better describe the features of road cameras. This can emphasize the video content features of specific locations, such as camera locations near intersections or congested areas, which helps to extract key features related to road traffic, thereby obtaining more comprehensive and accurate road camera features. In addition, by dot-multiplying the coordinate global correlation feature matrix and multiple monitoring key frame feature matrices by position, the dimension of the features can be expanded, which can improve the feature expression ability and more comprehensively describe the features of road cameras. Specifically, the final road camera feature matrix will contain information about the coordinate global correlation features and multiple monitoring key frame features, and the value at each position represents the degree of association of the position in different feature matrices.

[0027] In particular, in order to integrate and associate the traffic flow information of different sections in different time periods, so as to more comprehensively understand and analyze the traffic conditions, the technical solution of the present application extracts features of the traffic flow of each section in multiple predetermined time periods. It should be understood that the traffic flow of different sections in different time periods reflects the overall state of the traffic network. By extracting the traffic flow features and associating them to the global situation, the load conditions of each section, the degree of traffic congestion, etc. can be comprehensively considered to better understand the overall traffic conditions. In addition, by analyzing the changes in traffic flow in different time periods, traffic patterns such as peak periods and trough periods can also be identified, and traffic planning and optimization can be carried out accordingly. By extracting features of the traffic flow of each section in multiple predetermined time periods, the key features of each section can be extracted and integrated into the global correlation feature matrix of traffic flow, so as to better understand and analyze the traffic conditions.

[0028] Figure 2 The flowchart is a method for extracting features from the coordinate text data of each camera deployed on the road in a decision support information method based on data fusion analysis according to an embodiment of the present application. Figure 2As shown, in a specific embodiment of the present application, feature extraction is performed on the coordinate text data of each camera deployed on the road to obtain a coordinate global correlation feature matrix, including: S210, the coordinate text data of each camera deployed on the road are respectively passed through a semantic encoder to obtain a plurality of coordinate feature vectors; S220, the plurality of coordinate feature vectors are arranged in two dimensions to obtain a coordinate correlation feature matrix; S230, the coordinate correlation feature matrix is ​​passed through a coordinate feature extractor based on a convolutional neural network to obtain the coordinate global correlation feature matrix. It should be understood that coordinate text data is usually represented in text form, such as longitude and latitude or a specific road location description. However, text data is not convenient for calculation and analysis. Through a semantic encoder, the coordinate text data can be converted into a numerical vector representation, making it easier to process and compare. The semantic encoder can extract semantic and contextual information from the coordinate text data. The encoded vector can capture the similarities and differences between coordinates, thereby providing richer features for subsequent analysis. Through semantic encoding, the coordinate text data can be converted into a numerical vector representation, so that the coordinate text data of different cameras can be mapped to the same feature space, and then subsequent analysis and processing can be conveniently performed, thereby improving the availability and expression ability of the data. Specifically, the coordinate text data of each camera deployed on the road is segmented to obtain a word sequence; the embedding layer of the semantic encoder is used to map each word in the word sequence to a word embedding vector to obtain a sequence of word embedding vectors; the sequence of word embedding vectors is semantically encoded based on the global context using the converter-based Bert model of the semantic encoder to obtain multiple coordinate feature vectors. Encoding the coordinate text data of each camera deployed on the road by a semantic encoder is conducive to mapping the coordinate text of each camera deployed on the road into a similar high-dimensional feature space, so as to capture the high-dimensional implicit feature expression of the topological information of each camera.

[0029] Furthermore, by arranging the coordinate feature vectors into a matrix, the association and interaction between the coordinates can be captured, and matrix operations and analysis can be performed more conveniently, thereby revealing the spatial relationship and pattern between the coordinates. It should be understood that the coordinates represent the location information, and the coordinate association feature matrix can present the spatial relationship between different coordinates. By analyzing the matrix, spatial patterns such as distance, direction, and clustering between coordinates can be discovered, thereby inferring the characteristics of roads, geographical areas, or other spatial structures.

[0030] Furthermore, after the multiple coordinate feature vectors are arranged in two dimensions, a coordinate feature extractor based on a convolutional neural network is used to further extract and capture the global correlation information between the coordinates, so as to convert the coordinate correlation feature matrix into a more advanced and more expressive coordinate global correlation feature matrix. Convolutional neural networks have excellent feature extraction capabilities in the field of image processing, and can effectively capture local and global spatial relationships through convolution and pooling operations. Through multi-layer convolution and pooling operations, convolutional neural networks can gradually expand the receptive field, thereby capturing a wider range of coordinate correlation information. In this way, global correlation features between coordinates can be obtained, such as coordinate distribution and distribution patterns in the entire area. And by introducing a nonlinear activation function, nonlinear modeling can be performed on the coordinate correlation features. In this way, the correlation patterns between complex coordinates can be better captured, making the coordinate global correlation feature matrix more expressive. Specifically, each layer of the coordinate feature extractor based on the convolutional neural network is used to perform convolution processing, mean pooling processing and nonlinear activation processing on the input data in the forward pass of the layer so that the last layer of the coordinate feature extractor based on the convolutional neural network outputs the coordinate global correlation feature matrix, wherein the input of the coordinate feature extractor based on the convolutional neural network is the coordinate correlation feature matrix.

[0031] In a specific embodiment of the present application, feature extraction is performed on the multiple road monitoring videos collected by the multiple cameras deployed on the road to obtain multiple monitoring key frame feature matrices, including: extracting key frames from the multiple road monitoring videos collected by the multiple cameras deployed on the road to obtain multiple monitoring key frames; passing the multiple monitoring key frames through a monitoring feature extractor based on a three-dimensional convolutional neural network to obtain the multiple monitoring key frame feature matrices. It should be understood that road monitoring videos are usually continuous video streams, containing a large amount of redundant information, and processing the entire video sequence may require a long time and computing resources. By extracting key frames, video data can be compressed into a series of key frames to focus processing on the most important frames, thereby improving processing efficiency and reducing storage space and transmission bandwidth requirements. Specifically, key frames are usually the most representative and key information frames in the video. They can capture important events, actions or changes, such as traffic accidents, traffic flow changes, etc. By extracting key frames, it is easier to obtain key information in the video, that is, to filter out the most representative and key information frames. More specifically, key frames may be extracted by fixed-interval sampling, frame difference method, motion detection algorithm (such as optical flow method or background modeling), and the like.

[0032] Furthermore, the standard deep convolutional neural network uses a two-dimensional convolution kernel to perform convolution processing, pooling processing and activation processing on the input data layer by layer, and fully connects the feature map in the last layer of the convolutional neural network. However, since the video data contains a large number of image frames and there is a high degree of correlation in the temporal dimension between each image frame, if a standard deep convolutional neural network is used in the technical solution of the present application, on the one hand, the amount of calculation will be too large, and on the other hand, the performance of the standard deep convolutional neural network in extracting correlation features in the temporal dimension is not excellent. Therefore, in the technical solution of the present application, a monitoring feature extractor based on a three-dimensional convolutional neural network is used to encode the multiple monitoring key frames to extract the local high-dimensional implicit features in the multiple monitoring key frames and the correlation between the local high-dimensional implicit features of each monitoring key frame in the temporal dimension to obtain the feature matrix of the multiple monitoring key frames. Specifically, each layer of the monitoring feature extractor based on the three-dimensional convolutional neural network performs the following on the input data in the forward pass of the layer: uses a convolution module to perform three-dimensional convolution encoding on the input data with a three-dimensional convolution kernel to obtain a convolution feature map; uses a pooling module to perform pooling processing on the convolution feature map to obtain a pooled feature map; uses an activation module to perform nonlinear activation on the feature values ​​of each position of the pooled feature map to obtain an activated feature map; wherein the output of the last layer of the monitoring feature extractor based on the three-dimensional convolutional neural network is the multiple monitoring key frame feature matrices.

[0033] In a specific embodiment of the present application, feature extraction is performed on the traffic flow of each road section in multiple predetermined time periods to obtain the traffic flow global correlation feature matrix, including: passing the traffic flow of each road section in multiple predetermined time periods through a traffic flow feature extractor based on a sequence encoder to obtain multiple traffic flow feature vectors; feature encoding is performed on the multiple traffic flow feature vectors to obtain the traffic flow global correlation feature matrix. It should be understood that traffic flow data is usually collected in chronological order and contains traffic flow information of vehicles in different time periods. By using a sequence encoder, these time series features, such as traffic flow trends and periodic changes in each time period, can be automatically learned and extracted. The traffic flow value of each time period in the traffic flow data is associated with the traffic flow values ​​of the previous and next time periods. Models based on sequence encoders (such as recurrent neural networks, Transformers, etc.) can capture temporal dependencies and contextual information, thereby extracting more meaningful features. By converting traffic flow data into feature vectors, traffic flow data of different time periods and road sections can be mapped to the same feature space. Specifically, a sequence encoder model suitable for capturing the temporal relationship and long-term dependency in the time series can be selected, such as a recurrent neural network (RNN) and a long short-term memory network (LSTM), and then a deep learning framework (such as TensorFlow, PyTorch, etc.) is used to build a traffic flow feature extractor model based on a sequence encoder, which takes the traffic flow time series as input and outputs the corresponding traffic flow feature vector. In a specific embodiment of the present application, a sequence encoder including a one-dimensional convolutional layer and a fully connected layer is used to process the traffic flow of each road section in multiple predetermined time periods.

[0034] Furthermore, there may be complex correlations between different traffic flow features. For example, the traffic flow of certain sections may affect the traffic flow of other sections. For example, time features, spatial features, historical features, etc., will increase the computational complexity and may lead to the curse of dimensionality. Through feature encoding, high-dimensional feature vectors can be converted into low-dimensional feature representations to capture these global correlations, thereby better understanding the overall traffic flow situation and reducing the computational burden. Specifically, multiple traffic flow feature vectors can be aggregated and encoded using statistical feature extraction, neural network encoding, graph structure modeling, etc. to obtain the traffic flow global correlation feature matrix.

[0035] In a specific embodiment of the present application, the multiple traffic flow feature vectors are feature encoded to obtain the traffic flow global correlation feature matrix, including: two-dimensionally splicing the multiple traffic flow feature vectors to obtain the traffic flow correlation feature matrix; passing the traffic flow correlation feature matrix through a traffic flow correlation feature extractor based on a convolutional neural network to obtain the traffic flow global correlation feature matrix. It should be understood that each feature vector may contain different traffic flow features, such as vehicle speed, vehicle density, road type, etc. Through splicing, these features can be integrated into a feature matrix to retain their original information to more comprehensively describe various aspects of traffic flow. The spliced ​​feature matrix provides a global view that can simultaneously consider the relationship between different traffic flow feature vectors. For example, it can be observed whether there is a correlation between the vehicle speed at a certain location and the vehicle density at an adjacent location. This global view helps to discover potential patterns and associations in traffic flow data.

[0036] Furthermore, passing the traffic flow correlation feature matrix through a traffic flow correlation feature extractor based on a convolutional neural network can help extract the global correlation features of the traffic flow. By using the traffic flow correlation feature extractor based on a convolutional neural network, its convolution layer and pooling layer can be used to automatically learn the spatial structure and pattern of the traffic flow correlation features to combine them into a higher-level feature representation. Specifically, the traffic flow correlation feature matrix is ​​processed using the traffic flow correlation feature extractor based on a convolutional neural network with the following global correlation formula to obtain the traffic flow global correlation feature matrix;

[0037] Wherein, the global association formula is:

[0038] f i =active(N i ×f i-1 +B i )

[0039] Among them, f i-1 is the input of the i-th layer of the traffic flow correlation feature extractor based on the convolutional neural network, f i is the output of the i-th layer of the traffic flow correlation feature extractor based on the convolutional neural network, N i is the global transformation matrix of the i-th layer of the traffic flow correlation feature extractor based on the convolutional neural network, and B i is the bias vector of the i-th layer of the traffic flow correlation feature extractor based on the convolutional neural network, and active represents the nonlinear activation function.

[0040] In the above-mentioned decision support information method based on data fusion analysis, the step S130 predicts the traffic flow prediction value of the road section to be predicted at the next moment based on the road camera feature matrix and the vehicle flow global correlation feature matrix. It should be understood that the road camera can provide real-time traffic images or video data. By processing and analyzing these data, features related to traffic flow can be extracted. These features may include the number of vehicles, vehicle speed, vehicle density, etc. Integrating these features into the feature matrix can provide information about the current traffic conditions. The global correlation feature matrix obtained by the vehicle flow correlation feature extractor can capture the relationship and pattern between different vehicle flow features. These relationships and patterns may include mutual influence between vehicles, the spread of traffic congestion, etc. Taking into account the road camera feature matrix and the vehicle flow global correlation feature matrix, the mapping relationship from input features to output traffic flow prediction values ​​can be learned to predict the traffic flow prediction value of the road section at the next moment.

[0041] Figure 3 The flowchart is a method for predicting the traffic flow prediction value of the road section to be predicted at the next moment based on the road camera feature matrix and the vehicle flow global correlation feature matrix in the decision support information method based on data fusion analysis according to the embodiment of the present application. Figure 3As shown, in a specific embodiment of the present application, based on the road camera feature matrix and the traffic flow global correlation feature matrix, the traffic flow prediction value of the road section to be predicted at the next moment is predicted, including: S310, the road camera feature matrix and the traffic flow global correlation feature matrix are passed through the graph neural network to obtain a topological traffic flow feature matrix; S320, the topological traffic flow feature matrix is ​​expanded to obtain a topological traffic flow feature vector; S330, the topological traffic flow feature vector is optimized based on predictive learning matching viewpoint prediction to obtain an optimized topological traffic flow feature vector; S340, the optimized topological traffic flow feature vector is passed through a decoder to obtain a decoded value, and the decoded value is used to predict the traffic flow prediction value of the road section to be predicted at the next moment. It should be understood that the graph neural network can effectively process data with a graph structure, and the traffic network usually has the characteristics of a graph structure. In a traffic network, roads can be regarded as nodes, and the connections between roads can be regarded as edges. The road camera feature matrix provides local information of each node (road), and the traffic flow global correlation feature matrix provides global correlation information between nodes. Through graph neural networks, this information can be effectively combined and more complex traffic flow features can be learned. Specifically, graph neural networks can consider the connections and global structures between nodes while considering the local features of nodes. It can capture the dependencies and interaction patterns between nodes through message passing and feature aggregation between nodes. By performing multi-layer information transmission and feature extraction in graph neural networks, a topological traffic flow feature matrix can be obtained, in which each node represents a road section and the features of each node represent the traffic flow conditions of the road section. More specifically, the graph neural network processes the road camera feature matrix and the traffic flow global correlation feature matrix through learnable neural network parameters to obtain the topological traffic flow feature matrix.

[0042] Furthermore, the topological traffic flow characteristic matrix describes the traffic flow relationship between different roads in the traffic network. It is usually a two-dimensional matrix, in which each element represents the traffic flow between two roads. However, for some tasks and algorithms, this matrix needs to be converted into a vector form, namely the topological traffic flow characteristic vector. Common expansion methods include row expansion, column expansion, and expansion according to a certain rule.

[0043] In particular, in the technical solution of the present application, it is considered that if the camera coordinate data, surveillance video or traffic flow data are missing or erroneous, it will directly affect the accuracy of feature extraction. The semantic encoder may not be able to fully understand all the information in the coordinate text data, resulting in limited expression of the coordinate feature vector. The coordinate feature extractor and traffic flow association feature extractor based on convolutional neural networks may not be able to capture all the complex patterns related to traffic flow prediction. When the decoder converts the topological traffic flow feature matrix into a feature vector, it may not be able to accurately reflect all the information of the original data. This will lead to a problem in which the topological traffic flow feature vector will have a harmonious offset relative to the decoder when decoded by the decoder, affecting the decoding iteration effect. In order to solve this problem, in the technical solution of the present application, the topological traffic flow feature vector is optimized based on predictive learning matching to obtain an optimized topological traffic flow feature vector.

[0044] Among them, the topological traffic flow feature vector is optimized based on predictive learning matching to obtain an optimized topological traffic flow feature vector, including: performing probabilistic activation based on the Sigmoid function on the topological traffic flow feature vector to obtain a probabilistic topological traffic flow feature vector; inputting the topological traffic flow feature vector into the softmax function to obtain an initial class label probability value; calculating the difference between the feature value of each position in the probabilistic topological traffic flow feature vector and the initial class label probability value to obtain a difference feature vector, and passing the difference feature vector through a mask function to obtain a topological traffic flow viewpoint metric prediction initial weight vector; calculating the topological traffic flow viewpoint metric prediction initial weight vector The autocorrelation matrix of the topological traffic flow viewpoint metric prediction initial weight vector is obtained by dividing the autocorrelation matrix of the topological traffic flow viewpoint metric prediction initial weight vector by the square root of the length of the topological traffic flow viewpoint metric prediction initial weight vector, and then multiplying it by the topological traffic flow viewpoint metric prediction initial weight vector through the softmax function to obtain the topological traffic flow viewpoint metric prediction attention-oriented weight vector; the topological traffic flow viewpoint metric prediction attention-oriented weight vector is multiplied by the topological traffic flow feature vector according to the position point to obtain the first weighted topological traffic flow feature vector; the predetermined weight hyperparameter is multiplied by the topological traffic flow feature vector, and then added to the first weighted topological traffic flow feature vector according to the position to obtain the optimized topological traffic flow feature vector.

[0045] The topological vehicle flow feature vector is subjected to viewpoint prediction optimization based on predictive learning matching to obtain an optimized topological vehicle flow feature vector, including:

[0046] V cp =sigmoid(V co )

[0047]

[0048]

[0049]

[0050]

[0051] Among them, V co represents the topological traffic flow feature vector, sigmoid represents the activation function, V cp represents the probabilistic topological traffic flow feature vector, P represents the initial class label probability value, represents positional subtraction, mask represents the mask function, x represents the variable of the input mask function, θ represents the predetermined threshold, V e represents the initial weight vector for topological traffic flow viewpoint metric prediction, represents matrix multiplication, T represents the transpose of the vector, L represents the length of the initial weight vector for the topological traffic flow viewpoint metric prediction, Softmax represents the normalized exponential function, V ea represents the topological traffic flow viewpoint metric prediction attention-oriented weight vector, α represents the predetermined weight hyperparameter, ⊙ represents the multiplication by position, Indicates addition by position, V' co Represents the optimized topological traffic flow feature vector

[0052] That is, in the technical solution of the present application, the topological traffic flow feature vector is subjected to viewpoint prediction optimization based on predictive learning matching. First, the topological traffic flow feature vector is subjected to probabilistic activation based on the Sigmoid function to obtain a probabilistic topological traffic flow feature vector, and the topological traffic flow feature vector is input into the softmax function to obtain the initial class label probability value, and the viewpoint metric prediction contribution of each position in the standardized probabilistic topological traffic flow feature vector to the predetermined class probability label is initialized through inverse error analysis and masking.

[0053] Then, the viewpoint space of inverse optimization is expanded by using the autocorrelation attention representation of the initial weight vector predicted by the viewpoint metric of the topological traffic flow, and its sub-dimensional consistency is used as the attention-guided weight. The inverse consistency attention mechanism of the sub-dimensional distribution is applied to the topological traffic flow feature vector. In this way, the viewpoint metric prediction based on post-machine learning is used to perform learning predictive matching between the forward feature manifold and the backward class probability distribution, so that the optimized topological traffic flow feature vector has a relatively better distribution consistency with respect to the class probability distribution of the decoder, so as to improve its decoding accuracy through the decoder.

[0054] Furthermore, decoding the optimized topological traffic flow feature vector through a decoder to obtain a decoded value, and then using the decoded value to predict the traffic flow prediction value of the road section to be predicted at the next moment, is a common sequence prediction method, which is usually used for modeling and prediction of time series data. The basic idea of ​​this method is to capture the characteristics of the input sequence by establishing an encoder-decoder model and generate the corresponding output sequence. In this case, the input sequence is the optimized topological traffic flow feature vector, and the output sequence is the traffic flow prediction value of the road section to be predicted at the next moment.

[0055] In summary, the embodiment of the present application first obtains the coordinate text data of each camera deployed on the road, multiple road monitoring videos collected by multiple cameras deployed on the road, and the traffic flow of each road section in multiple predetermined time periods, and then uses deep learning technology to perform feature extraction and correlation analysis on the three, and finally obtains the decoded value through the decoder to predict the traffic flow prediction value of the road section to be predicted at the next moment, thereby providing more comprehensive and accurate information for traffic management and decision making, thereby helping to improve transportation efficiency and the quality of urban transportation.

[0056] Figure 4 FIG. 1 is a block diagram of a decision support information system based on data fusion analysis according to an embodiment of the present application. Figure 4 As shown, according to the decision support information system 100 based on data fusion analysis according to the embodiment of the present application, it includes: a traffic data acquisition module 110, which is used to acquire the coordinate text data of each camera deployed on the road, a plurality of road monitoring videos collected by a plurality of cameras deployed on the road, and the traffic flow of each road section in a plurality of predetermined time periods, wherein the coordinate text data includes the X-axis coordinate, Y-axis coordinate and Z-axis coordinate of each camera; a traffic data feature extraction module 120, which is used to extract a road camera feature matrix and a traffic flow global correlation feature matrix from the coordinate text data of each camera deployed on the road, the plurality of road monitoring videos collected by a plurality of cameras deployed on the road, and the traffic flow of each road section in a plurality of predetermined time periods; a traffic flow prediction generation module 130, which is used to predict the traffic flow prediction value of the road section to be predicted at the next moment based on the road camera feature matrix and the traffic flow global correlation feature matrix.

[0057] As described above, the decision support information system 100 based on data fusion analysis according to the embodiment of the present application can be implemented in various terminal devices, such as a server deployed with a decision support information control algorithm based on data fusion analysis. In one example, the decision support information system 100 based on data fusion analysis can be integrated into the terminal device as a software module and / or a hardware module. For example, the decision support information system 100 based on data fusion analysis can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the decision support information system 100 based on data fusion analysis can also be one of the many hardware modules of the terminal device.

[0058] Alternatively, in another example, the decision support information system 100 based on data fusion analysis and the terminal device may also be separate devices, and the decision support information system 100 based on data fusion analysis may be connected to the terminal device via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0059] Here, those skilled in the art can understand that the specific operations of each step in the above-mentioned decision support information system based on data fusion analysis have been referred to above. Figures 1 to 3 The description of the decision support information method based on data fusion analysis has been introduced in detail, and therefore, its repeated description will be omitted.

[0060] An embodiment of the present application provides an electronic device. Figure 5 is a block diagram of an electronic device according to an embodiment of the present application.

[0061] like Figure 5 As shown, the electronic device 10 includes an input device 11, an input interface 12, a central processing unit 13, a memory 14, an output interface 15, an output device 16, and a bus 17. The input interface 12, the central processing unit 13, the memory 14, and the output interface 15 are connected to each other through the bus 17, and the input device 11 and the output device 16 are connected to the bus 17 through the input interface 12 and the output interface 15, respectively, and then connected to other components of the electronic device 10.

[0062] Specifically, the input device 11 receives input information from the outside and transmits the input information to the central processing unit 13 through the input interface 12; the central processing unit 13 processes the input information based on the computer executable instructions stored in the memory 14 to generate output information, temporarily or permanently stores the output information in the memory 14, and then transmits the output information to the output device 16 through the output interface 15; the output device 16 outputs the output information to the outside of the electronic device 10 for user use.

[0063] In one embodiment, Figure 5 The electronic device 10 shown can be implemented as a network device, which may include: a memory configured to store programs; a processor configured to run the programs stored in the memory to execute any one of the decision support information methods based on data fusion analysis described in the above embodiments.

[0064] According to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program tangibly contained on a machine-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network, and / or installed from a removable storage medium.

[0065] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0066] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present application, but the present application is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and substance of the present application, and these modifications and improvements are also considered to be within the scope of protection of the present application.

Claims

1. A decision support information method based on data fusion analysis, characterized in that: include: Obtaining coordinate text data of each camera deployed on the road, multiple road monitoring videos collected by multiple cameras deployed on the road, and traffic flow of each road section in multiple predetermined time periods, wherein the coordinate text data includes the X-axis coordinate, Y-axis coordinate, and Z-axis coordinate of each camera; Extracting a road camera feature matrix and a traffic flow global correlation feature matrix from the coordinate text data of each camera deployed on the road, a plurality of road monitoring videos collected by the plurality of cameras deployed on the road, and the traffic flow of each road section in a plurality of predetermined time periods; Based on the road camera feature matrix and the vehicle flow global correlation feature matrix, the traffic flow prediction value of the road section to be predicted at the next moment is predicted.

2. The decision support information method based on data fusion analysis according to claim 1 is characterized in that: A road camera feature matrix and a traffic flow global correlation feature matrix are extracted from the coordinate text data of each camera deployed on the road, the multiple road monitoring videos collected by the multiple cameras deployed on the road, and the traffic flow of each road section in multiple predetermined time periods, including: Performing feature extraction on the coordinate text data of each camera deployed on the road to obtain a coordinate global correlation feature matrix; Performing feature extraction on the plurality of road monitoring videos collected by the plurality of cameras deployed on the road to obtain a plurality of monitoring key frame feature matrices; Multiplying the coordinate global correlation feature matrix and the plurality of monitoring key frame feature matrices by position points to obtain the road camera feature matrix; The vehicle flow of each road section in a plurality of predetermined time periods is subjected to feature extraction to obtain the vehicle flow global correlation feature matrix.

3. The decision support information method based on data fusion analysis according to claim 2 is characterized in that: Feature extraction is performed on the coordinate text data of each camera deployed on the road to obtain a coordinate global correlation feature matrix, including: Passing the coordinate text data of each camera deployed on the road through a semantic encoder to obtain a plurality of coordinate feature vectors; Arranging the plurality of coordinate feature vectors in two dimensions to obtain a coordinate correlation feature matrix; The coordinate correlation feature matrix is ​​passed through a coordinate feature extractor based on a convolutional neural network to obtain the coordinate global correlation feature matrix.

4. The decision support information method based on data fusion analysis according to claim 3 is characterized in that: Performing feature extraction on the plurality of road monitoring videos collected by the plurality of cameras deployed on the road to obtain a plurality of monitoring key frame feature matrices includes: Extracting key frames from the plurality of road monitoring videos collected by the plurality of cameras deployed on the road to obtain a plurality of monitoring key frames; The multiple monitoring key frames are passed through a monitoring feature extractor based on a three-dimensional convolutional neural network to obtain the multiple monitoring key frame feature matrices.

5. The decision support information method based on data fusion analysis according to claim 4 is characterized in that: Extracting features of the traffic flow of each road section in a plurality of predetermined time periods to obtain the traffic flow global correlation feature matrix includes: The traffic flow of each road section in a plurality of predetermined time periods is passed through a traffic flow feature extractor based on a sequence encoder to obtain a plurality of traffic flow feature vectors; Feature encoding is performed on the multiple vehicle flow feature vectors to obtain the vehicle flow global correlation feature matrix.

6. The decision support information method based on data fusion analysis according to claim 5 is characterized in that: Performing feature encoding on the plurality of vehicle flow feature vectors to obtain the vehicle flow global correlation feature matrix includes: Performing two-dimensional splicing on the plurality of vehicle flow feature vectors to obtain a vehicle flow correlation feature matrix; The vehicle flow correlation feature matrix is ​​passed through a vehicle flow correlation feature extractor based on a convolutional neural network to obtain the vehicle flow global correlation feature matrix.

7. The decision support information method based on data fusion analysis according to claim 6 is characterized in that: Based on the road camera feature matrix and the vehicle flow global correlation feature matrix, predicting the traffic flow prediction value of the road section to be predicted at the next moment, including: The road camera feature matrix and the vehicle flow global correlation feature matrix are passed through a graph neural network to obtain a topological vehicle flow feature matrix; Expanding the topological traffic flow characteristic matrix to obtain a topological traffic flow characteristic vector; Performing viewpoint prediction optimization based on predictive learning matching on the topological traffic flow feature vector to obtain an optimized topological traffic flow feature vector; The optimized topological traffic flow feature vector is passed through a decoder to obtain a decoded value, and the decoded value is used to predict the traffic flow prediction value of the road section to be predicted at the next moment.

8. The decision support information method based on data fusion analysis according to claim 7 is characterized in that: The topological traffic flow feature vector is subjected to viewpoint prediction optimization based on predictive learning matching to obtain an optimized topological traffic flow feature vector, including: Performing probabilistic activation based on the Sigmoid function on the topological traffic flow feature vector to obtain a probabilistic topological traffic flow feature vector; Input the topological traffic flow feature vector into a softmax function to obtain an initial class label probability value; Calculate the difference between the feature value of each position in the probabilistic topological traffic flow feature vector and the initial class label probability value to obtain a difference feature vector, and pass the difference feature vector through a mask function to obtain an initial weight vector for topological traffic flow viewpoint metric prediction; Calculate the autocorrelation matrix of the initial weight vector of the topological traffic flow viewpoint metric prediction, and divide the autocorrelation matrix of the initial weight vector of the topological traffic flow viewpoint metric prediction by the square root of the length of the initial weight vector of the topological traffic flow viewpoint metric prediction, and then multiply it by the initial weight vector of the topological traffic flow viewpoint metric prediction through a softmax function to obtain the attention-oriented weight vector of the topological traffic flow viewpoint metric prediction; Multiplying the topological traffic flow viewpoint metric prediction attention-oriented weight vector by the topological traffic flow feature vector by position point to obtain a first weighted topological traffic flow feature vector; The predetermined weight hyperparameter is multiplied by the topological traffic flow feature vector, and then the resultant is added to the first weighted topological traffic flow feature vector by position to obtain an optimized topological traffic flow feature vector.

9. A decision support information system based on data fusion analysis, characterized in that: include: A traffic flow data acquisition module, used to acquire coordinate text data of each camera deployed on the road, multiple road monitoring videos collected by multiple cameras deployed on the road, and traffic flow of each road section in multiple predetermined time periods, wherein the coordinate text data includes the X-axis coordinate, Y-axis coordinate, and Z-axis coordinate of each camera; A traffic flow data feature extraction module, used to extract a road camera feature matrix and a traffic flow global correlation feature matrix from the coordinate text data of each camera deployed on the road, the multiple road monitoring videos collected by the multiple cameras deployed on the road, and the traffic flow of each road section in multiple predetermined time periods; The traffic flow prediction generation module is used to predict the traffic flow prediction value of the road section to be predicted at the next moment based on the road camera feature matrix and the traffic flow global correlation feature matrix.

10. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the decision support information method based on data fusion analysis according to any one of claims 1 to 8 when executing the computer program.

Images