Crowd movement prediction method based on generative adversarial networks

Abstract

The present invention discloses a crowd movement prediction method based on a generative adversarial network to study the problem of spatiotemporal feature fusion of multi-modal data, taking the prediction of urban crowd flow during the COVID-19 epidemic as a research object. Due to the influence of complex multi-modal data such as complex social background, policies, epidemic situation, etc., it is very difficult to predict crowd movement patterns during the epidemic period. The patent analyzed crowd movement data of three cities in China and found that although there is a large gap between different cities, their crowd movement patterns during the epidemic period of the COVID-19 epidemic show high similarity. Based on this, the patent designed a prediction model based on a conditional generative adversarial network and fused and modeled the impact of multi-modal data on crowd movement spatiotemporal features in the model. In addition, the model can help the government to better evaluate the potential impact of various policies on crowd mobility and optimize policy formulation.

Description

[Technical field]

[0001] The present invention belongs to the field of crowd movement prediction, and particularly relates to a crowd movement prediction method based on a generative adversarial network, which can help understand how crowd movement patterns change during the COVID-19 epidemic, and in particular, the movement restriction policies issued by the government during the epidemic and related statistical data (e.g., confirmed cases) show the discipline that led crowd movement during the novel coronavirus-related pneumonia pandemic, and accurately predict the crowd movement level. [Background technology]

[0002] Understanding how crowd movement patterns change during the COVID-19 epidemic is crucial for containing the epidemic. Considering the influence of factors such as complex social backgrounds, differences in individual behavior, and extremely limited data, crowd movement patterns during the epidemic appear to be unpredictable.

[0003] Without accurate prediction of crowd movement discipline, it is difficult to formulate rational policies. Overly strict epidemic prevention policies will have a relatively large impact on the economy, while overly lax epidemic prevention policies will also make it difficult to control the epidemic and affect people's lives and health. Therefore, accurate predictions are necessary to provide accurate reference for policy formulation.

[0004] The current pattern of solving urban crowd movement prediction based on deep learning models is mainly to train based on abundant historical data, and then, given a certain period of observation data in the current and historical time, the model can output deterministic prediction results. However, such a method has certain limitations. First, the deterministic prediction model cannot generate a probability estimate of crowd mobility, and the generated results are relatively fixed, and it is not easy to flexibly change some conditions to perform multi-dimensional simulation. During the epidemic period of novel coronavirus-related pneumonia, government decision makers need more flexible models, such as generative models, which can learn the distribution of data to simulate multiple potential crowd movement disciplines under different policies, and the government can formulate a phased reopening plan based on the simulated crowd movement response results during the epidemic period. Therefore, the applicant has provided a conditional generative adversarial network for complex dynamic modeling between newly confirmed cases, policies and crowd movement. Summary of the Invention [Problem to be solved by the invention]

[0005] The purpose of the present invention is to complete and standardize the deficiencies in existing research and technology, and provide a crowd movement prediction method based on generative adversarial networks, which can accurately predict the change trend of crowd movement by modeling the impact of different policies and epidemic conditions during the epidemic period, which is more practical, and is favorable for analyzing the potential policy impact according to the results of data prediction, and the application of the method is highly scalable. [Means for solving the problem]

[0006] The object of the present invention is achieved by the following technical solutions:

[0007] A crowd movement prediction method based on generative adversarial networks is Step 1: Divide a city into H × W meshes of equal area, each mesh representing one region of the city; Step 2 divides the area in step 1 and collects statistics on the crowd movement level m of each area; Using the regional crowd movement level statistics obtained in step 2, we create a crowd movement heatmap M ∈ R consisting of one region in the city. H×W Step 3: obtain the matrix, where each element represents the crowd movement level of the corresponding area; Step 4: collect daily statistical data and related policies of different districts during the epidemic period, obtain the newly confirmed cases C every day as the representative statistical data of the epidemic period, obtain the changes and intensity of policies every day, and denote the intensity variable of these policies as P; Historical and current crowd movement heatmaps for a specific city t-1 ,M t} and corresponding statistical data on the new coronavirus-related pneumonia epidemic {C t-1 ,C t ,C t+1} and policy {P t-1 ,P t } Given a crowd movement level heat map for a certain time in the future, {M t+1} and step 5 of predicting

[0008] In addition, in step 5, a population flow pattern during the epidemic period is predicted by a crowd movement prediction model during the epidemic period, and a crowd movement level heat map is generated for a certain period of time in the future, where the crowd movement prediction model during the epidemic period includes a generator module, a discriminator module, and a domain knowledge fusion module; The generator module predicts a crowd movement discipline for a future period of time based on historical crowd movement data for a past period of time; The discriminator module is used to predict the label of the population flow heat map and determine whether the generated population flow heat map is consistent with a true population flow distribution; The domain knowledge fusion module is used to fuse the influence of external factors during the epidemic.

[0009] Furthermore, the generator module models the response of the population flow intensity of different regions during the epidemic period to policy changes by modeling the change values ​​of population flow during different time steps, and the input of the generator module can be expressed as the change in crowd movement level between two consecutive time slices: ΔM t-1 =M t -M t-1 .

[0010] In addition, a transformer-based encoder module is introduced into the epidemic crowd movement prediction model to model long-distance spatiotemporal correlation, and a multi-head self-attention mechanism module is introduced to extract and generate feature maps.

number

number

[0011] In addition, the domain knowledge fusion module uses the epidemic period policies and epidemic statistical data as conditions to fuse with the spatiotemporal crowd movement features. Specifically, a fully connected neural network is introduced to integrate different types of domain knowledge into hidden variables.

number

number

number

number

[0012] In addition, a mask matrix is ​​added to the crowd movement prediction model during the epidemic period.

number

number

number

number

[0013] Further, the daily policy includes one or more indicators of travel restrictions, lockdown policies, economic policies, and sanitation system policies. Effect of the Invention

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0015] 1) The present invention adopts an incremental learning method and can process rapidly changing population flow data during epidemics; 2) The present invention introduces a domain knowledge fusion module to model the impact of policies and epidemic developments on population flow changes; 3) The generator module and the classifier module in the present invention form a generative adversarial network to model uncertainty in the population flow process. [Brief description of the drawings]

[0016] [Figure 1] 2 is a flowchart of the present invention. [Diagram 2] FIG. 1 is a structural schematic diagram of the crowd movement prediction model during an epidemic in the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] In describing the present invention, it is necessary to understand that the orientations or positional relationships indicated by the terms "one end", "the other end", "outside", "up", "inside", "horizontal", "coaxial", "center", "end", "length", "outer end", etc. are orientations or positional relationships indicated based on the drawings, and are only for the purpose of easily explaining and simplifying the present invention, and are not intended to indicate or imply that the devices or elements referred to necessarily have a specific orientation or are constructed and operated in a specific orientation, and therefore should not be understood to limit the present invention.

[0018] The invention will now be further described in connection with the accompanying drawings, in which:

[0019] Referring to FIG. 1, a crowd movement prediction method based on a generative adversarial network includes: Step 1: Divide a city into H × W meshes of equal area, each mesh representing one region of the city; Step 2 divides the area in step 1 and collects statistics on the crowd movement level m of each area; Using the regional crowd movement level statistics obtained in step 2, we create a crowd movement heatmap M ∈ R consisting of one region in the city. H×WStep 3: obtain the matrix, where each element represents the crowd movement level of the corresponding area; Step 4: collect daily statistical data and related policies of different districts during the epidemic period, obtain the newly confirmed cases C every day as the representative statistical data of the epidemic period, obtain the changes and intensity of policies every day, and denote the intensity variable of these policies as P; Historical and current crowd movement heatmaps for a specific city t-1 ,M t} and corresponding statistical data on the new coronavirus-related pneumonia epidemic {C t-1 ,C t ,C t+1}(A large number of articles have already demonstrated accurate real-time prediction of newly confirmed cases) and policy {P t-1 ,P t } Given a heat map of crowd movement levels for a given time in the future, {M t+1} and step 5 of predicting

[0020] Further, in step 5, the population flow discipline during the epidemic period is predicted by the crowd movement prediction model during the epidemic period, and a crowd movement level heat map for a certain period of time in the future is generated, where the crowd movement prediction model during the epidemic period includes a generator module, a discriminator module, and a domain knowledge fusion module, where the generator module predicts the crowd movement discipline for a certain period of time in the future based on historical crowd movement data for a certain period of time in the past, the discriminator module is used to predict the label of the population flow heat map and determine whether the generated population flow heat map is consistent with the true population flow distribution, and the domain knowledge fusion module is used to fuse the influence of external factors during the epidemic period.

[0021] Among them, the generator module and the discriminator module constitute a generative adversarial network and are used to model uncertainty in the population flow process.

[0022] The epidemic period policy and statistical conditions such as the number of newly diagnosed cases are combined to perform crowd mobility estimation. The present invention provides that a model based on a deep generative network can be used to predict population flow discipline during the epidemic period. The generative model can learn the distribution of data to simulate multiple potential crowd flow disciplines under different policies, and the government can formulate a phased reopening plan based on the simulated crowd mobility response results during the epidemic period. The present invention provides a conditional generative adversarial network for complex dynamic modeling between newly confirmed cases, policies and crowd mobility. Unlike traditional models that model long-term timing dependencies, the model focuses on predicting crowd mobility transformations between adjacent time periods formed from the latest dynamics of potential epidemic-related statistical data, policies and crowd mobility. Specifically, the present invention designs a Policy-Human Mobility Interplay Network (PHMIN) model to estimate crowd mobility changes. Here, the conditional input of the model is mainly from statistical data and policies during the epidemic period. The model can flexibly learn fine-grained crowd movement dynamics and be accurately extended to predict multiple epidemics between cities. The model structure is shown in Figure 1.

[0023] Furthermore, the generator module efficiently models the response of the population flow intensity of different regions during the epidemic period to policy changes by modeling the change value of population flow between different time steps, and the input of the generator module can be expressed as the change in crowd movement level between two consecutive time slices: ΔM t-1 =M t -M t-1 .

[0024] Furthermore, in the epidemic period crowd movement prediction model, on the other hand, multi-scale spatio-temporal crowd movement features are considered, where a transformer-based encoder module is introduced to model long-distance spatio-temporal correlation, and a multi-head self-attention mechanism module is introduced to extract and generate feature maps.

number

number

[0025] In addition, the domain knowledge fusion module improves the accuracy of prediction by fusing the spatiotemporal crowd movement features with the epidemic period policies and epidemic statistical data as conditions. Specifically, a fully connected neural network is introduced to integrate different kinds of domain knowledge into the hidden variables

number

number

number

number

[0026] In addition, a mask matrix is ​​added to the model optimization process of the crowd movement prediction model during the epidemic.

number

number

number

number

[0027] Further, the daily policy includes one or more indicators of travel restrictions, lockdown policies, economic policies, and sanitation system policies.

[0028] Working Example In this embodiment, the above-mentioned epidemic period crowd movement prediction method is adopted to predict the crowd movement situation in three cities, Beijing, Dalian, and Shijiazhuang, during the epidemic period, where BJ stands for Beijing, DL stands for Dalian, SJZ stands for Shijiazhuang, and the latter number indicates the crowd movement data in different epidemic periods. In this embodiment, considering that seven new coronavirus-related pneumonia epidemics have occurred in the above three cities, the epidemic is divided into two intensities according to the actual situation, and three sets of experiments are obtained: 1) experiments with similar intensities between different cities, 2) experiments with different intensities in different cities, and 3) experiments with similar or different intensities in the same city, and the results are analyzed hierarchically. These experiments are designed to answer the fundamental question of whether the model can effectively generalize to different environments to predict the impact of policies on crowd flow when the next epidemic arrives. This issue will be discussed in detail below.

[0029] [Table 1]

[0030] Similar Intensity Across Different Cities: From the empirical results shown in Table 6.1, the following conclusions can be drawn:

[0031] Overall, the PHMIN model achieves the best performance compared to the baseline models, proving that the model has good generalization ability in the context of inter-city crowd movement prediction. Specifically, the relatively simple HA and ARIMA models outperform DeepST and cGAN, which to some extent means that the most valuable information in the problem is correlated with the latest situation of crowd flow. In addition, cGAN is slightly better than DeepST because it can model the unknown elements of crowd flow in a generative manner. On the other hand, DeepST shows deterministic prediction based on abundant historical observations, which is less suitable in epidemic scenarios due to the lack of historical data.

[0032] Given the same test city as the prediction target, we found that the models obtained based on different city training sets have different representations. When testing on DL1, SJZ1, and SJZ2, the model based on BJ1 training obviously performed better than the other cities. This can be attributed to Beijing's higher data quality, more available samples, and higher population density.

[0033] The similarity of the geographical distribution of cities may affect the effectiveness of the model's inter-city prediction. Experimental results show that models based on DL1 training usually yield worse performance, which is not much different from traditional ARIMA (better MAE, worse MAPE) when taking the results on DL1-BJ1, DL1-SJZ1, and DL1-SJZ2 as examples. In fact, Dalian city is surrounded by the sea on all sides and its geographical distribution is not very regular, while Beijing and Shijiazhuang have very regular, checkerboard-like urban areas. This leads us to speculate that the gap in spatial distribution may make the model more difficult to generalize between cities.

[0034] [Table 2]

[0035] Different Intensities in Different Cities: To study the generalization expression of the model during epidemics of different intensities, we further designed and conducted the experiments shown in Table 6.2, and the results of the experiments illustrated the following:

[0036] In most cases, the PHMIN model outperforms the baseline model. However, for the same test city, the overall performance predicted by the training set with different intensities is slightly worse than the model obtained from similar intensities in Table 6.1. This indicates that epidemic intensity is a more important indicator affecting the generalization ability of the model compared to different city layouts.

[0037] The model trained on the data of Dalian city is still not as good in other cities (e.g., DL1-BJ2 and DL2-SJZ2), which reiterates the importance of similar geographical distribution of cities. Moreover, due to the dual differences in geographical distribution and epidemic intensity, the prediction results of the provided model in the DL2-SJZ2 scenario are worse than the baseline method ARIMA.

[0038] [Table 3]

[0039] Similar or different strengths in the same city: Intuitively, if we run experiments in the same city, the model should generalize better and produce better results. However, the experimental data in Table 6.3 shows some different results:

[0040] First, the PHMIN model outperforms the baseline model in all experiments, and the three sets of experiments, SJZ1-SJZ2, DL2-DL3, and BJ1-BJ2, achieve better effects compared with the results in Tables 6.1 and 6.2, which means that the negative impact of gaps within the same city may be smaller than the effect of epidemic intensity.

[0041] However, two sets of experiments, DL1-DL2 and DL1-DL3, showed anomalies in the results, which were worse than BJ2-DL2 and BJ2-DL3, respectively. This can be attributed to BJ2, DL2, and DL3, the three datasets are in different cities, but experienced similar epidemic intensities. Comparing the results between DL2-DL3 and DL1-DL3, it can be inferred that the provided model can exhibit higher generalization performance when performing crowd flow prediction in scenes with similar epidemic intensities within the same city.

[0042] Finally, the above embodiments are only used to explain the technical solutions of the present invention, and are not limited thereto. The present invention has been described in detail with reference to the above embodiments. However, those skilled in the art may modify the technical solutions described in the above embodiments or equally replace some or all of the technical features thereof, and it should be understood that such modifications or replacements do not cause the essence of the corresponding technical solutions to depart from the scope of the technical solutions of the embodiments of the present invention.

[0043] (Additional Note) (Appendix 1) Step 1: Divide a city into H × W meshes of equal area, each mesh representing one region of the city; Step 2 divides the area in step 1 and collects statistics on the crowd movement level m of each area; Using the regional crowd movement level statistics obtained in step 2, we create a crowd movement heatmap M ∈ R consisting of one region in the city. H×W Step 3: obtain the matrix, where each element represents the crowd movement level of the corresponding area; Step 4: collect daily statistical data and related policies of different districts during the epidemic period, obtain the newly confirmed cases C every day as the representative statistical data of the epidemic period, obtain the changes and intensity of policies every day, and denote the intensity variable of these policies as P; Historical and current crowd movement heatmaps for a specific city t-1 ,M t} and corresponding statistical data on the new coronavirus-related pneumonia epidemic {C t-1 ,C t ,C t+1} and policy {P t-1 ,P t } Given a crowd movement level heat map for a certain time in the future, {M t+1} and step 5 of predicting the crowd movement based on the generative adversarial network.

[0044] (Appendix 2) In step 5, a population flow pattern during an epidemic is predicted by a crowd movement prediction model during an epidemic period, and a crowd movement level heat map is generated for a certain period of time in the future. The crowd movement prediction model during an epidemic period includes a generator module, a discriminator module, and a domain knowledge fusion module; The generator module predicts a crowd movement discipline for a future period of time based on historical crowd movement data for a past period of time; The discriminator module is used to predict the label of the population flow heat map and determine whether the generated population flow heat map is consistent with a true population flow distribution; The crowd movement prediction method based on generative adversarial networks described in Appendix 1, characterized in that the domain knowledge fusion module is used to fuse the influence of external factors during the epidemic period.

[0045] (Appendix 3) The generator module models the response of population flow intensity of different regions during the epidemic period to policy changes by modeling the change value of population flow during different time steps, and the input of the generator module can be expressed as the change in crowd movement level between two consecutive time slices: ΔM t-1 =M t -M t-1 3. A crowd movement prediction method based on a generative adversarial network as described in claim 2.

[0046] (Appendix 4) A transformer-based encoder module is introduced into the epidemic crowd movement prediction model to model long-distance spatiotemporal correlation, and a multi-head self-attention mechanism module is introduced to extract and generate feature maps.

number

number

[0047] (Appendix 5) The domain knowledge fusion module uses the epidemic period policies and epidemic statistical data as conditions to fuse with the spatiotemporal crowd movement characteristics. Specifically, a fully connected neural network is introduced to integrate different types of domain knowledge into hidden variables.

number

number

number

number

[0048] (Appendix 6) A mask matrix for the epidemic period crowd movement prediction model

number

number

number

number

[0049] (Appendix 7) The crowd movement prediction method based on the generative adversarial network described in Appendix 1, characterized in that the daily policy includes one or more indicators of travel restrictions, lockdown policies, economic policies, and sanitation system policies.

Claims

1. A crowd movement prediction method based on a generative adversarial network, in which a computer executes the following steps: Step 1, partitioning a city into H x W meshes of equal area, each mesh representing a region of the city; Step 2 of dividing the area in step 1 and collecting statistics on the crowd movement level m of each of the different areas; Using the regional crowd movement level statistics obtained in step 2, a crowd movement heat map M ∈ R is constructed for one region in the city. Ｈ×Ｗ Step 3: obtaining a matrix of the crowd movement level of the corresponding area; Step 4: collect daily statistical data and related policies of different regions during the epidemic period, obtain the newly confirmed cases C every day as the representative statistical data of the epidemic period, obtain the changes and intensity of policies every day, and denote the intensity variable of these policies as P; Historical and current crowd movement heatmaps for a specific city ｔ－１ ,M ｔ } and corresponding statistical data on the new coronavirus-related pneumonia epidemic {C ｔ－１ ,C ｔ ,C ｔ＋１ } and policy {P ｔ－１ , Pt}, and the crowd movement level heat map {M ｔ＋１ }.}.

2. In step 5, a population flow pattern during an epidemic is predicted by a crowd movement prediction model during an epidemic period, and a crowd movement level heat map is generated for a certain period of time in the future. The crowd movement prediction model during an epidemic period includes a generator module, a discriminator module, and a domain knowledge fusion module; The generator module predicts a crowd movement discipline for a future period of time based on historical crowd movement data for a past period of time; The discriminator module is used to predict the label of the population flow heat map and determine whether the generated population flow heat map is consistent with a true population flow distribution; The crowd movement prediction method based on generative adversarial network according to claim 1, characterized in that the domain knowledge fusion module is used to fuse the influence of external factors during an epidemic.

3. The generator module models the response of population flow intensity of different regions during the epidemic period to policy changes by modeling the change value of population flow during different time steps, and the input of the generator module can be expressed as the change in crowd movement level between two consecutive time slices: ΔM ｔ－１ = M ｔ -M ｔ－１ The crowd movement prediction method according to claim 2, characterized in that:

4. A transformer-based encoder module is introduced into the epidemic crowd movement prediction model to model long-distance spatiotemporal correlation, and a multi-head self-attention mechanism module is introduced to extract and generate feature maps. [0010] Then, the features after transformer processing are [0025] The crowd movement prediction method based on the generative adversarial network according to claim 2, further comprising: combining the output features with the external condition features and conveying them to a decoder to output the predicted population flow result.

5. The domain knowledge fusion module uses the epidemic period policies and epidemic statistical data as conditions to fuse with the spatiotemporal crowd movement characteristics. Specifically, a fully connected neural network is introduced to integrate different types of domain knowledge into hidden variables. [0030] , and then introduce one gate-control fusion network module to deactivate the spatiotemporal features of different regions: [0045] The crowd movement prediction model for epidemic periods operates with a single noise vector [0050] and spatiotemporal feature vectors are simultaneously introduced and joined on the feature dimension. Finally, the crowd movement prediction model during the epidemic period involves introducing a connection between the models to obtain an estimate for the level of population movement in the next time step: [006] The crowd movement prediction method according to claim 2, characterized in that:

6. A mask matrix for the epidemic period crowd movement prediction model [0070] We can introduce a mask matrix to reduce the effect of regions with missing samples and compute the loss function with the mask matrix from the generator module: [0080] The loss function of the resulting classifier module is: [0090] Finally, we combine the loss functions of the generator and discriminator modules to obtain: [0089] The crowd movement prediction method based on a generative adversarial network according to claim 2, characterized in that a saddle point of the loss functions of the generator module and the discriminator module is reached by training the model, indicating that model training is completed.

7. The crowd movement prediction method based on a generative adversarial network according to claim 1 , wherein the daily policy includes one or more indicators of travel restrictions, lockdown policies, economic policies, and sanitation system policies.

Images