A pedestrian flow prediction method based on integrated graph convolutional network based on multimodal fusion

Abstract

A method for predicting pedestrian flow based on an integrated graph convolutional network with multimodal fusion, which relates to the field of intelligent traffic management and control technology. S1. Acquire and process the pedestrian flow heat map data and environmental data in hot spots, and splice the heat map data with the environmental data to achieve multimodal fusion; S2. Establish a fully connected graph network in the hot spots to obtain an adjacency matrix; S3. Design an integrated graph convolutional network based on multimodal fusion as a pedestrian flow prediction model; S4. Use the heat map features of the next moment as the label of the previous moment, train the model, and adjust the model parameters; S5. Use the trained model to predict the pedestrian flow heat map at future moments, and display the prediction results in a visual way. Using the pedestrian flow heat map data and environmental data at historical moments, predict the pedestrian flow heat map data at future moments, and display the prediction results in a visual way, the prediction results are more accurate and intuitive.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent traffic management and control, and specifically to a method for predicting passenger flow using an integrated graph convolutional network based on multimodal fusion. Background Art

[0002] With the continuous acceleration of urbanization, problems such as urban population mobility and traffic congestion have become increasingly prominent, bringing new challenges to intelligent traffic management and resource allocation. Traditional pedestrian flow prediction methods often rely on only a single data source or feature, such as using only traffic flow data or only using population density data, which leads to the need to improve the accuracy and stability of the prediction results. A single data source may not be able to fully reflect the complex changes in pedestrian flow, especially in urban traffic management and planning, which requires comprehensive consideration of multiple data sources to obtain accurate prediction results. Therefore, there is an urgent need for a method that can comprehensively utilize multiple data sources to improve the accuracy and stability of pedestrian flow prediction.

[0003] In addition, traditional methods also have limitations in the visualization of prediction results, and often cannot intuitively show the spatial distribution of pedestrian flow and its changing trends, which brings difficulties to urban traffic management and resource allocation. With the support of modern information technology, it has become possible to use visualization tools such as maps to show the spatial distribution of pedestrian flow, but how to combine the prediction results with these visualization tools to intuitively show the distribution characteristics and changing trends of pedestrian flow is still a problem worth studying. Summary of the invention

[0004] In order to address the shortcomings of the background technology, the present invention provides an integrated graph convolutional network pedestrian flow prediction method based on multimodal fusion, which uses the pedestrian flow heat map data and environmental data at historical moments to predict the pedestrian flow heat map data at future moments, and displays the prediction results in a visual way, making the prediction results more accurate and intuitive.

[0005] To achieve the above object, the present invention adopts the following technical solution: a method for predicting pedestrian flow based on an integrated graph convolutional network with multimodal fusion, comprising the following steps:

[0006] S1. Obtain and process the heat map data and environmental data of the hotspot area, and splice the heat map data with the environmental data to achieve multimodal fusion;

[0007] The hotspot area is an area selected within a specific time period according to the needs. The heat map data and environmental data at T moments are obtained according to the needs. The processing of the heat map data and the environmental data specifically includes: converting the heat map from multi-dimensional data to one-dimensional data through the Flatten layer, expressed as a matrix X G ∈R (N×D), N represents the number of nodes, D represents the characteristic dimension of the heat map; the environmental data is represented by data, represented as a matrix X E ∈R (N×F) , F represents the environmental feature dimension;

[0008] The processed thermal map data and environmental data are spliced ​​to achieve multimodal fusion and obtain the feature matrix X F ∈R [N×(D+F)] , where X F =[X G ,X E ];

[0009] All features are normalized to obtain the feature matrix X, which is calculated as follows:

[0010]

[0011] Where, X F is the original value of the feature matrix, min(X F ) is the feature matrix X F The minimum value in max(X F ) is the feature matrix X F The maximum value in ;

[0012] S2. Establish a fully connected graph network of the hotspot area and obtain the adjacency matrix A;

[0013] Among them, the fully connected graph network means that each node is connected to all other nodes in the network, so its adjacency matrix A is a matrix with all 1s and the elements on the diagonal are 0. Suppose the fully connected graph network has N nodes, then the element A in the adjacency matrix A is (i,j) It is expressed as follows:

[0014]

[0015] The adjacency matrix A is a symmetric matrix. When node i is connected to node j, node j is also connected to node i.

[0016] S3. Design an integrated graph convolutional network based on multimodal fusion as a pedestrian flow prediction model;

[0017] Among them, the structure of the graph convolutional network includes two graph convolutional layers, a batch normalization layer, a residual connection, a dropout layer and a fully connected layer;

[0018] The feature matrix X and adjacency matrix A at different times are respectively input into the graph convolution network to obtain the output of a single base model. First, it passes through the first graph convolution layer, aggregates the information from adjacent nodes through graph convolution, and projects it into a new feature space to obtain a new feature matrix X (1) , retain the output of the first layer as res(X(1) ) is used in the residual connection, the specific formula is as follows:

[0019]

[0020] In the formula, σ represents the activation function, using the Relu function. express The degree matrix of represents the adjacency matrix A plus the adjacency matrix after self-connection, X (0) is the original feature matrix, W (1) is the weight matrix of the first convolutional layer;

[0021] Add a batch normalization layer between the first graph convolution layer and the Relu function for processing;

[0022] Next, the feature matrix X (1) The adjacency matrix A enters the second graph convolution layer to further aggregate information from adjacent nodes and project them into the same feature space to obtain a new feature matrix X (2) , the specific formula is as follows:

[0023]

[0024] Where W (2) is the weight matrix of the second convolutional layer;

[0025] Then enter the residual connection and convert the feature matrix X output by the second graph convolution layer (2) Connect with residual res(X (1) ) add to get the feature matrix X (2)′ , the specific formula is as follows:

[0026] X (2) ′=X (2) +res(X (1) )

[0027] The nonlinear function Relu is introduced to enhance the expressiveness of the model. After the discard layer, the features of some nodes are randomly discarded during the training process to prevent overfitting, and the feature matrix X is obtained. (2)″ ;

[0028] Then, the obtained feature matrix X (2)″ After the fully connected layer, the node representation is projected to the required output dimension. The specific formula is as follows:

[0029] X (2)″′ =X (2)″ W (3)

[0030] Where W (3) is the weight matrix of the fully connected layer;

[0031] Finally, X is transformed into (2)″′ Compress it to between 0 and 1 to get the predicted value Y of the base model. The specific formula is as follows:

[0032]

[0033] Where e is the base of natural logarithm;

[0034] The integrated learning uses the Bagging algorithm to learn the graph convolutional network and calculate the average value of all graph convolutional network outputs. The final integrated graph convolutional network based on multimodal fusion is used as the pedestrian flow prediction model. The prediction results are shown as follows:

[0035]

[0036] In the formula, Represents the predicted value of the traffic prediction model, K represents the number of base models, and the base models include GCN 1 ,GCN 2 …,GCN K , Y k represents the output prediction value of the base model, k = 1, 2, ..., K;

[0037] S4, using the heat map features of the next moment as the label of the previous moment, training the model, and adjusting the model parameters;

[0038] The label for the t-th moment is designed as follows: divide the heat map at time t+1 into blocks in the form of a matrix, extract the traffic density level of each block, and then flatten it into a one-dimensional vector as the label y of the node at the t-th moment t , so that the model learns the feature matrix X at the current moment and the output label y at the next moment t The mapping relationship between them;

[0039] In the forward propagation stage of the model, Smooth L1 Loss is selected as the loss function of the model to calculate the error between the model prediction value and the true label. The calculation formula is as follows:

[0040]

[0041] In the formula, y represents the true label;

[0042] In the back-propagation phase, the Adam optimization algorithm is selected to calculate the gradient of the model parameters and update the parameters to minimize the loss function. The parameters include the weight matrix W of the first convolutional layer. (1) , the weight matrix W of the second convolutional layer (2) and the weight matrix W of the fully connected layer (3); Set the initial value of the learning rate, and then dynamically adjust the learning rate according to the gradient of each parameter. The calculation formula of the Adam optimization algorithm is as follows:

[0043]

[0044] In the formula, p represents the number of iterations, θ p+1 represents the updated parameters, θ p represents the parameter at the pth iteration, η represents the learning rate, represents the expected gradient, represents the gradient variance, ∈ represents the numerical stability term;

[0045] The pedestrian flow prediction model is trained for several iterations. In each training: for each input feature matrix X and adjacency matrix A, the output is calculated through forward propagation, and then the loss function value is calculated. The loss function is back-propagated. According to the gradient information, the Adam optimization algorithm updates the model parameters. The above steps are repeated until the training of all input feature matrices X and adjacency matrices A is completed;

[0046] S5. Use the trained model to predict the heat map of future traffic flow and display the prediction results in a visual way;

[0047] When predicting, use the trained traffic prediction model to generate predicted values The predicted value Converted to an image, where each predicted value Corresponding to a pixel value in the image, different pixel values ​​are mapped to corresponding labels and represent different levels of pedestrian flow density. The pedestrian flow density is divided into multiple levels from low to high and distinguished by different colors. Finally, a visualization image is drawn.

[0048] Furthermore, in step S3, the discard rate of the discard layer is set to 0.5.

[0049] Furthermore, the initial value of the learning rate in step S4 is set to 0.001.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] 1. The present invention uses multimodal fusion to integrate multiple data sources, combines the heat map data of human flow and environmental data for prediction, and introduces integrated learning, which helps to improve the accuracy and stability of the prediction results. At the same time, the Bagging algorithm is used to integrate the results of multiple base models, and the overall prediction performance is improved by average integration;

[0052] 2. The present invention displays the prediction results in a visual way, intuitively showing the spatial distribution and changing trends of passenger flow, improving the comprehensibility of the prediction results, and providing an important reference for urban traffic management and resource allocation. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 is a flow chart of the present invention;

[0054] Figure 2 It is a schematic diagram of the model of the graph convolutional network in the present invention;

[0055] Figure 3 It is a flow chart of integrated learning in the present invention. DETAILED DESCRIPTION

[0056] The technical solution of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0057] like Figure 1 to Figure 3 As shown in the figure, a method for predicting pedestrian flow based on an integrated graph convolutional network with multimodal fusion is proposed. Figure 1 As shown, the following steps are included:

[0058] S1. Obtain and process the heat map data and environmental data of the hotspot area, and splice the heat map data with the environmental data to achieve multimodal fusion;

[0059] The hotspot area is an area selected within a specific time period according to demand. It can be selected from areas with relatively concentrated or dense traffic, which attract a large number of people for various reasons, such as commercial centers, tourist attractions, transportation hubs, entertainment venues, etc. The heat map data can be a screenshot of the Baidu Map Geographic Information Platform. The heat map is divided into seven colors (corresponding to seven levels) according to the density of the crowd. The environmental data includes weather, wind, air quality, etc., which can be selected as needed according to specific needs.

[0060] The processing of heat map data and environmental data specifically includes: converting the heat map from multi-dimensional data to one-dimensional data through the Flatten layer, represented as a matrix X G ∈R (N×D) , where N represents the number of nodes and D represents the feature dimension of the heat map. The environmental data is represented by data, expressed as a matrix X E ∈R (N×F), where F represents the environmental feature dimension. The above heat map data and environmental data are the features of one moment, and the features of T moments are obtained according to the requirements.

[0061] The processed thermal map data and environmental data are spliced ​​to achieve multimodal fusion and obtain the feature matrix X F ∈R [N×(D+F)] , where X F =[X G ,X E ].

[0062] All features are normalized to obtain the feature matrix X, which is calculated as follows:

[0063]

[0064] Where, X F is the original value of the feature matrix, min(X F ) is the feature matrix X F The minimum value in max(X F ) is the feature matrix X F The maximum value in .

[0065] S2. Establish a fully connected graph network of the hotspot area and obtain the adjacency matrix A;

[0066] The fully connected graph network means that each node is connected to all other nodes in the network, so its adjacency matrix A is a matrix with all 1s and the elements on the diagonal are 0, indicating that the node is not connected to itself. Assuming that the fully connected graph network has N nodes, the element A in the adjacency matrix A is (i,j) It is expressed as follows:

[0067]

[0068] The adjacency matrix A of a fully connected graph network is a symmetric matrix because the connections are bidirectional, that is, when node i is connected to node j, node j is also connected to node i.

[0069] S3. Design an integrated graph convolutional network based on multimodal fusion as a pedestrian flow prediction model;

[0070] Among them, the structure of the graph convolutional network contains two graph convolutional layers, a batch normalization layer, a residual connection, a dropout layer and a fully connected layer.

[0071] Combination Figure 2 As shown in Figure 1, the feature matrix X and adjacency matrix A at different times are input into the graph convolution network to obtain the output of a single base model. First, after the first graph convolution layer, the information from adjacent nodes is aggregated through graph convolution and projected into a new feature space to obtain a new feature matrix X. (1), retain the output of the first layer as res(X (1) ) in order to be used in the residual connection, the specific formula is as follows:

[0072]

[0073] In the formula, σ represents the activation function, using the Relu function. express The degree matrix of represents the adjacency matrix A plus the adjacency matrix after self-connection, X (0) is the original feature matrix, W (1) is the weight matrix of the first convolutional layer.

[0074] In order to stabilize the training process and accelerate convergence, a batch normalization layer is added between the first graph convolution layer and the ReLU function.

[0075] Next, the feature matrix X (1) The adjacency matrix A enters the second graph convolution layer to further aggregate information from adjacent nodes and project them into the same feature space to obtain a new feature matrix X (2) , the specific formula is as follows:

[0076]

[0077] Where W (2) is the weight matrix of the second convolutional layer.

[0078] Then enter the residual connection and convert the feature matrix X output by the second graph convolution layer (2) Connect with residual res(X (1) ) add to get the feature matrix X (2)′ , to enhance the gradient flow and prevent the gradient from disappearing. The specific formula is as follows:

[0079] X (2) ′=X (2) +res(X (1) )

[0080] The nonlinear function Relu is introduced to enhance the expressiveness of the model. After a dropout layer, the dropout rate is set to 0.5. During the training process, the features of some nodes are randomly discarded to prevent overfitting, and the feature matrix X is obtained. (2)″ .

[0081] Then, the obtained feature matrix X (2)″ After a fully connected layer, the node representation is projected to the required output dimension. The specific formula is as follows:

[0082] X (2)″′ =X (2)″ W (3)

[0083] Where W (3) is the weight matrix of the fully connected layer.

[0084] Finally, X is transformed into (2)″′ Compress it to between 0 and 1 to get the predicted value Y of the base model. The specific formula is as follows:

[0085]

[0086] Where e is the base of natural logarithms.

[0087] Ensemble learning uses the Bagging algorithm, which aims to improve the overall prediction performance and enhance the robustness and generalization ability of the model by integrating the results of multiple base classifiers through bootstrap sampling and voting or averaging. This paper uses the Bagging algorithm to learn the graph convolutional network, calculates the average value of all graph convolutional network outputs, and combines Figure 3 As shown in the figure, the training set is randomly sampled and divided into multiple training subsets as the input of each graph convolutional network. The prediction performance is improved by integrating the results of multiple base models. The final integrated graph convolutional network based on multimodal fusion is used as the pedestrian flow prediction model. The prediction results are shown as follows:

[0088]

[0089] In the formula, Represents the predicted value of the traffic prediction model, K represents the number of base models, and the base models include GCN 1 ,GCN 2 …,GCN K , Y k Represents the output prediction value of the base model, k=1,2,…,K.

[0090] S4, using the heat map features of the next moment as the label of the previous moment, training the model, and adjusting the model parameters;

[0091] In supervised learning, in addition to the predicted output of the crowd flow prediction model, labels are also required as target variables to guide the model training process and evaluate the performance of the trained model. There are T data points in total, and the label for the tth moment is designed as follows: divide the heat map at time t+1 into blocks in the form of a matrix (such as 4×4), extract the crowd flow density level of each block, and then flatten it into a one-dimensional vector as the label y of the node at the tth moment t By using the heat map features of the next moment as labels, the problem is transformed into a supervised learning task, so that the model can learn the feature matrix X at the current moment and the output label y at the next moment.t The mapping relationship between them.

[0092] In the forward propagation stage of the model, Smooth L1 Loss is selected as the loss function of the model to calculate the error between the model prediction value and the true label. Smooth L1 Loss has a smoother gradient at outliers and is more robust to outliers, which is conducive to improving the robustness and generalization ability of the model. Smooth L1 Loss is an approximation of Huber Loss, and its calculation formula is as follows:

[0093]

[0094] Where y represents the true label.

[0095] In the back-propagation phase, the Adam optimization algorithm is selected to calculate the gradient of the model parameters and update the parameters to minimize the loss function, including the weight matrix W of the first convolutional layer. (1) , the weight matrix W of the second convolutional layer (2) and the weight matrix W of the fully connected layer (3) The initial value of the learning rate is set to 0.001, and then the learning rate is dynamically adjusted according to the gradient of each parameter to improve the efficiency and stability of training. Adam is an optimization algorithm with adaptive learning rate, and the calculation formula is as follows:

[0096]

[0097] In the formula, p represents the number of iterations, θ p+1 represents the updated parameters, θ p represents the parameter at the pth iteration, η represents the learning rate, represents the gradient expectation (momentum), represents the gradient variance (RMSprop), ∈ represents the numerical stability term to prevent division by zero.

[0098] The pedestrian flow prediction model is trained for several (for example, 100) iterations. In each training: for each input feature matrix X and adjacency matrix A, the output is calculated through forward propagation, and then the loss function value is calculated. The loss function is back-propagated. According to the gradient information, the Adam optimization algorithm updates the model parameters. The above steps are repeated until the training of all input feature matrices X and adjacency matrices A is completed, so that the pedestrian flow prediction model can be gradually optimized on the training data and fit the distribution characteristics of the data, thereby realizing accurate prediction of pedestrian flow.

[0099] S5. Use the trained model to predict the heat map of future traffic flow and display the prediction results in a visual way;

[0100] When predicting, use the trained traffic prediction model to generate predicted values The predicted value Converted to an image, where each predicted value Corresponding to a pixel value in the image, different pixel values ​​are mapped to corresponding labels and represent different levels of pedestrian flow density. The pedestrian flow density is divided into multiple levels from low to high and distinguished by different colors. Finally, a visualization image is drawn.

[0101] Example

[0102] The heat map data and environmental data of the 14 historical moments of the day in the hot spots of Harbin are obtained with a preset time interval of 1h as the step length, providing basic data for the subsequent prediction of the flow of people. The hot spots include 9 areas, namely Harbin North Station, Harbin Music Park, Harbin West Station, Harbin Station, Chinese Baroque Street, Harbin South Station, Labor Park, Wanda Plaza (Xiangfang), and Harbin East Station. The heat map data of the flow of people comes from the screenshots of the Baidu Map Geographic Information Platform. The size of each image is 100×100. The heat map of urban flow of people is divided into seven colors (i.e., seven levels) according to the density of the crowd. The environmental data includes the temperature and wind force level per hour.

[0103] In order to more intuitively illustrate the effect of the prediction method of the present invention, an experiment was conducted under the above conditions to predict the heat map of the flow of people at the next moment and compare it with the existing model. The results are shown in Table 1 below:

[0104]

[0105]

[0106] It can be seen that the method of the present invention has good prediction performance.

[0107] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other forms of assembly without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered in all respects as exemplary and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations within the meaning and range of equivalents of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0108] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A method for predicting pedestrian flow based on an integrated graph convolutional network with multimodal fusion, characterized by: The following steps are involved: S1. Obtain and process the heat map data and environmental data of the hotspot area, and splice the heat map data with the environmental data to achieve multimodal fusion; The hotspot area is an area selected within a specific time period according to the needs. The heat map data and environmental data at T moments are obtained according to the needs. The processing of the heat map data and the environmental data specifically includes: converting the heat map from multi-dimensional data to one-dimensional data through the Flatten layer, expressed as a matrix X G ∈R (N×D) , N represents the number of nodes, D represents the characteristic dimension of the heat map; the environmental data is represented by data, represented as a matrix X E ∈R (N×F) , F represents the environmental characteristic dimension; The processed thermal map data and environmental data are spliced ​​to achieve multimodal fusion and obtain the feature matrix X F ∈R [N ×(D+F)] , where X F =[X G ,X E ]; All features are normalized to obtain the feature matrix X, which is calculated as follows: Where, X F is the original value of the feature matrix, min(X F ) is the feature matrix X F The minimum value in max(X F ) is the feature matrix X F The maximum value in ; S2. Establish a fully connected graph network of the hotspot area and obtain the adjacency matrix A; Among them, the fully connected graph network means that each node is connected to all other nodes in the network, so its adjacency matrix A is a matrix with all 1s and the elements on the diagonal are 0. Suppose the fully connected graph network has N nodes, then the element A in the adjacency matrix A is (i,j) It is expressed as follows: The adjacency matrix A is a symmetric matrix. When node i is connected to node j, node j is also connected to node i. S3. Design an integrated graph convolutional network based on multimodal fusion as a pedestrian flow prediction model; Among them, the structure of the graph convolutional network includes two graph convolutional layers, a batch normalization layer, a residual connection, a dropout layer and a fully connected layer; The feature matrix X and adjacency matrix A at different times are respectively input into the graph convolution network to obtain the output of a single base model. First, it passes through the first graph convolution layer, aggregates the information from adjacent nodes through graph convolution, and projects it into a new feature space to obtain a new feature matrix X (1) , retain the output of the first layer as res(X (1) ) is used in the residual connection, the specific formula is as follows: In the formula, σ represents the activation function, using the Relu function. express The degree matrix of represents the adjacency matrix A plus the adjacency matrix after self-connection, X (0) is the original feature matrix, W (1) is the weight matrix of the first convolutional layer; Add a batch normalization layer between the first graph convolution layer and the Relu function for processing; Next, the feature matrix X (1) The adjacency matrix A enters the second graph convolution layer to further aggregate information from adjacent nodes and project them into the same feature space to obtain a new feature matrix X (2) , the specific formula is as follows: Where W (2) is the weight matrix of the second convolutional layer; Then enter the residual connection and convert the feature matrix X output by the second graph convolution layer (2) Connect with residual res(X (1) ) add to get the feature matrix X (2)′ , the specific formula is as follows: X (2) ′=X (2) +res(X (1) ) The nonlinear function Relu is introduced to enhance the expressiveness of the model. After the discard layer, the features of some nodes are randomly discarded during the training process to prevent overfitting, and the feature matrix X is obtained. (2)″ ; Then, the obtained feature matrix X (2)″ After the fully connected layer, the node representation is projected to the required output dimension. The specific formula is as follows: X (2) ″′=X (2) ″W (3) Where W (3) is the weight matrix of the fully connected layer; Finally, X is transformed into (2)″′ Compress it to between 0 and 1 to get the predicted value Y of the base model. The specific formula is as follows: Where e is the base of natural logarithm; The integrated learning uses the Bagging algorithm to learn the graph convolutional network and calculate the average value of all graph convolutional network outputs. The final integrated graph convolutional network based on multimodal fusion is used as the pedestrian flow prediction model. The prediction results are shown as follows: In the formula, represents the predicted value of the traffic prediction model, K represents the number of base models, and the base models include GCN1, GCN2…, GCN K , Y k represents the output prediction value of the base model, k = 1, 2, ..., K; S4, using the heat map features of the next moment as the label of the previous moment, training the model, and adjusting the model parameters; The label for the t-th moment is designed as follows: divide the heat map at time t+1 into blocks in the form of a matrix, extract the traffic density level of each block, and then flatten it into a one-dimensional vector as the label y of the node at the t-th moment t , so that the model learns the feature matrix X at the current moment and the output label y at the next moment t The mapping relationship between them; In the forward propagation stage of the model, Smooth L1 Loss is selected as the loss function of the model to calculate the error between the model prediction value and the true label. The calculation formula is as follows: In the formula, y represents the true label; In the back-propagation phase, the Adam optimization algorithm is selected to calculate the gradient of the model parameters and update the parameters to minimize the loss function. The parameters include the weight matrix W of the first convolutional layer. (1) , the weight matrix W of the second convolutional layer (2) and the weight matrix W of the fully connected layer (3) ; Set the initial value of the learning rate, and then dynamically adjust the learning rate according to the gradient of each parameter. The calculation formula of the Adam optimization algorithm is as follows: In the formula, p represents the number of iterations, θ p+1 represents the updated parameters, θ p represents the parameter at the pth iteration, η represents the learning rate, represents the expected gradient, represents the gradient variance, ∈ represents the numerical stability term; The pedestrian flow prediction model is trained for several iterations. In each training: for each input feature matrix X and adjacency matrix A, the output is calculated through forward propagation, and then the loss function value is calculated. The loss function is back-propagated. According to the gradient information, the Adam optimization algorithm updates the model parameters. The above steps are repeated until the training of all input feature matrices X and adjacency matrices A is completed; S5. Use the trained model to predict the heat map of future traffic flow and display the prediction results in a visual way; When predicting, use the trained traffic prediction model to generate predicted values The predicted value Converted to an image, where each predicted value Corresponding to a pixel value in the image, different pixel values ​​are mapped to corresponding labels and represent different levels of pedestrian flow density. The pedestrian flow density is divided into multiple levels from low to high and distinguished by different colors. Finally, a visualization image is drawn.

2. According to claim 1, a method for predicting pedestrian flow based on an integrated graph convolutional network with multimodal fusion, characterized in that: In step S3, the discard rate of the discard layer is set to 0.

5.

3. The method for predicting pedestrian flow based on an integrated graph convolutional network using multimodal fusion according to claim 1 is characterized in that: The initial value of the learning rate in step S4 is set to 0.001.

Images