People flow prediction method and device, electronic equipment and nonvolatile storage medium

Abstract

The application discloses a kind of crowd flow prediction method, device, electronic equipment and nonvolatile storage medium.Therein, the method includes: determining the grid area corresponding to target scenic spot, wherein the grid area includes grid unit, each grid unit corresponds a scenic spot in target scenic spot;According to the video image corresponding to grid unit, determine the crowd space-time parameter of target unit in grid unit in first time period;Using target model, according to crowd space-time parameter, obtain the crowd flow of target unit in second time period, wherein the second time period is the next time period of first time period.This application solves the technical problem that the related art can not specifically predict the crowd flow of each scenic spot in the scenic area when counting the crowd flow of tourists in the scenic area, only based on the data of the entrance guard of the scenic area, resulting in low crowd flow prediction accuracy and poor tourist experience.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and more specifically, to a method, apparatus, electronic device, and non-volatile storage medium for predicting pedestrian traffic. Background Technology

[0002] With the continuous development of the national economy, the tourism industry has also become increasingly developed. However, during holidays, tourists often encounter problems such as difficulty in planning travel routes, difficulty in choosing among numerous attractions, traffic jams before entering attractions, and excessive crowds after entering scenic areas, which seriously affect the tourist experience. On the scenic area side, there are also issues with inaccurate control of tourist numbers. When there are many tourists, it is difficult to provide timely warnings and make corresponding arrangements to avoid safety hazards; when there are few tourists, the inability to adjust in time may lead to a waste of resources and manpower.

[0003] When conducting visitor flow statistics in scenic areas, the relevant technologies only analyze data from the scenic area's access control system, and cannot specifically predict the visitor flow at each attraction within the scenic area, resulting in technical problems such as low accuracy in visitor flow prediction and poor visitor experience.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This application provides a method, apparatus, electronic device, and non-volatile storage medium for predicting visitor flow, in order to at least solve the technical problem that related technologies, when counting visitor flow in scenic areas, only analyze data from the scenic area's access control system and cannot specifically predict the visitor flow at each attraction within the scenic area, resulting in low accuracy in visitor flow prediction and a poor visitor experience.

[0006] According to one aspect of the embodiments of this application, a method for predicting pedestrian flow is provided, comprising: determining a grid area corresponding to a target scenic spot, wherein the grid area includes grid cells, each grid cell corresponding to a scenic spot in the target scenic spot; determining the spatiotemporal parameters of pedestrian flow of a target cell in a first time period based on video images corresponding to the grid cells, wherein the spatiotemporal parameters of pedestrian flow are used to characterize at least the pedestrian flow of the target cell in the first time period, and the flow of people between the target cell and other grid cells, the target cell being the grid cell for which pedestrian flow prediction is planned; and using a target model to predict the pedestrian flow of the target cell in a second time period based on the spatiotemporal parameters of pedestrian flow, wherein the second time period is the time period immediately following the first time period, the target model being a model trained using historical spatiotemporal parameter data, the historical spatiotemporal parameter data including the spatiotemporal parameters of pedestrian flow of each grid cell in multiple historical time periods.

[0007] Optionally, determining the spatiotemporal parameters of pedestrian flow in the target unit within the first time period based on the video image corresponding to the grid unit includes: determining the spatiotemporal parameters of pedestrian flow in the target unit by identifying the target human figures at various times in the video image. The spatiotemporal parameters of pedestrian flow include: pedestrian flow, crowd activity flow, crowd input flow, and crowd output flow. Pedestrian flow is used to characterize the number of people passing through the target unit within the first time period. Crowd activity flow is used to characterize the number of people transferring from the target unit to the other grid units within the first time period. Crowd input flow is used to characterize the number of people flowing into the target unit within the first time period. Crowd output flow is used to characterize the number of people flowing from the target unit to the other grid units within the first time period.

[0008] Optionally, using a target model to predict based on spatiotemporal parameters of pedestrian flow includes: determining the grayscale value of grid cells in a grid area based on pedestrian flow to obtain a target flow map, wherein the grayscale value is used to characterize the pedestrian flow size corresponding to the grid cell; generating a target interaction flow map based on pedestrian activity flow, pedestrian input flow, and pedestrian output flow; extracting a first spatial feature from the target flow map and a second spatial feature from the target interaction flow map through a convolutional neural network in the target model; extracting target temporal features corresponding to the first and second spatial features in a first time period based on the gated loop control unit in the target model, and then fusing the first spatial feature, the second spatial feature, and the target temporal feature and inputting them into the fully connected layer of the target model for prediction.

[0009] Optionally, identifying the target human figure image at each moment in the video image includes: performing motion target detection on each frame of the video image to obtain a motion target region; performing human figure contour recognition processing on the motion target region to obtain a human figure candidate region, wherein the human figure contour recognition processing includes at least: clumping processing, area filtering, and aspect ratio filtering; and using a human figure classifier model to classify and identify the human figure candidate region to obtain the target human figure image, wherein the human figure classifier model is a model trained with preset sample data, and the preset sample data includes: first sample data that has been identified as a human figure candidate region after human figure contour recognition processing but is not actually a target human figure image, and second sample data that has been identified as a human figure candidate region after human figure contour recognition processing and is a target human figure image.

[0010] Optionally, after obtaining the target human figure image, the method further includes: performing similarity matching between the target human figure image and a preset clothing image to obtain a target similarity; if the target similarity is greater than a preset similarity threshold, the target human figure image is identified as a staff member image, wherein the staff member image is not included in the number of people in the process of determining the spatiotemporal parameters of the pedestrian flow.

[0011] Optionally, the method further includes: acquiring weather forecast data corresponding to the second time period; determining a first identifier and a second identifier corresponding to the second time period, wherein the first identifier is used to characterize the season corresponding to the second time period, and the second identifier is used to characterize whether the second time period is a holiday; and using a target model, at least based on the weather forecast data, the first identifier, and the second identifier, predicting the flow of people in the target unit during the second time period.

[0012] Optionally, after obtaining the visitor flow of the target unit in the second time period, the method further includes: determining the estimated queuing time of the corresponding scenic spot of the target unit based on the visitor flow of the target unit in the second time period, and sending the estimated queuing time to the display device in the target scenic area for display; and sending corresponding alarm prompt information when the visitor flow exceeds the preset flow threshold.

[0013] According to another aspect of the embodiments of this application, a pedestrian flow prediction device is also provided, comprising: a grid division module, used to determine a grid area corresponding to a target scenic spot, wherein the grid area includes grid cells, and each grid cell corresponds to a scenic spot in the target scenic spot; a parameter determination module, used to determine the spatiotemporal parameters of pedestrian flow of a target cell in a grid cell within a first time period based on video images corresponding to the grid cells, wherein the spatiotemporal parameters of pedestrian flow are used to characterize at least the pedestrian flow of the target cell within the first time period, and the flow of people between the target cell and other grid cells, and the target cell is a grid cell for which pedestrian flow prediction is planned; and a pedestrian flow prediction module, used to use a target model to predict the pedestrian flow of the target cell within a second time period based on the spatiotemporal parameters of pedestrian flow, wherein the second time period is the time period immediately following the first time period, and the target model is a model trained using historical spatiotemporal parameter data, the historical spatiotemporal parameter data including the spatiotemporal parameters of pedestrian flow of each grid cell in multiple historical time periods.

[0014] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory and a processor, the processor being configured to run a program stored in the memory, wherein the program executes a pedestrian flow prediction method during runtime.

[0015] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored computer program, wherein the device where the non-volatile storage medium is located executes a people flow prediction method by running the computer program.

[0016] In this embodiment, a grid area corresponding to the target scenic area is determined, wherein the grid area includes grid cells, and each grid cell corresponds to a scenic spot in the target scenic area; based on the video images corresponding to the grid cells, the spatiotemporal parameters of the pedestrian flow in the target cell within the grid cell are determined in a first time period, wherein the spatiotemporal parameters of the pedestrian flow are used to characterize at least the pedestrian flow in the target cell within the first time period, as well as the pedestrian flow between the target cell and other grid cells, and the target cell is the grid cell for which pedestrian flow prediction is planned; using a target model, based on the spatiotemporal parameters of the pedestrian flow, the pedestrian flow in the target cell within a second time period is predicted, wherein the second time period is the time immediately following the first time period. The target model is trained using historical spatiotemporal parameter data, which includes the spatiotemporal parameters of pedestrian flow for each grid cell across multiple historical time periods. By deploying cameras at various scenic spots and using human figure recognition and pedestrian flow prediction technologies, the model infers the distribution of pedestrian flow at each scenic spot and displays the real-time data on the scenic area's large screen and management app. This achieves the goal of accurately predicting pedestrian flow at each scenic spot, thus solving the technical problem of low accuracy and poor visitor experience caused by related technologies that only analyze data from the scenic area's access control system and cannot specifically predict the pedestrian flow at each scenic spot. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 This is a hardware structure block diagram of a computer terminal (or electronic device) for implementing a method for predicting pedestrian flow, according to an embodiment of this application.

[0019] Figure 2 This is a schematic diagram of a method for predicting pedestrian traffic according to an embodiment of this application;

[0020] Figure 3a This is a schematic diagram of a crowd movement flow according to an embodiment of this application;

[0021] Figure 3b This is a schematic diagram of a crowd input stream provided according to an embodiment of this application;

[0022] Figure 3c This is a schematic diagram of a crowd output stream provided according to an embodiment of this application;

[0023] Figure 4 This is a schematic diagram of the architecture of a target model provided according to an embodiment of this application;

[0024] Figure 5 This is a schematic diagram of model training according to an embodiment of this application;

[0025] Figure 6 This is a schematic diagram of a pedestrian flow prediction device provided according to an embodiment of this application. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] In related technologies, when counting visitor flow in scenic areas, analysis is based solely on data from the scenic area's access control system, making it impossible to specifically predict visitor flow at each attraction within the scenic area. Therefore, this results in low accuracy in visitor flow prediction and a poor visitor experience. To address this issue, this application provides a relevant solution, which is detailed below.

[0029] According to an embodiment of this application, a method for predicting pedestrian traffic is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0030] The methods and embodiments provided in this application can be executed on mobile terminals, computer terminals, or similar computing devices. Figure 1 A hardware block diagram of a computer terminal (or electronic device) for implementing a pedestrian flow prediction method is shown. Figure 1 As shown, the computer terminal 10 (or electronic device 10) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0031] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or electronic device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0032] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the pedestrian flow prediction method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the aforementioned pedestrian flow prediction method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0033] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0034] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or electronic device).

[0035] Under the above operating environment, this application provides a method for predicting pedestrian traffic. Figure 2 This is a schematic diagram of a method for predicting pedestrian traffic according to an embodiment of this application, as shown below. Figure 2 As shown, the method includes the following steps:

[0036] Step S202: Determine the grid area corresponding to the target scenic area, wherein the grid area includes grid cells, and each grid cell corresponds to a scenic spot in the target scenic area;

[0037] To facilitate the expression of spatial relationships, this application abstracts each area (scenic spot) within the scenic area into regular grid areas, with each grid unit representing a scenic spot area within the scenic area.

[0038] Step S204: Based on the video images corresponding to the grid cells, determine the spatiotemporal parameters of the pedestrian flow in the target cell within the first time period. The spatiotemporal parameters of the pedestrian flow are used to characterize the pedestrian flow in the target cell within the first time period and the pedestrian flow between the target cell and other grid cells. The target cell is the grid cell for which pedestrian flow prediction is planned.

[0039] In the technical solution provided in step S204, video images of each grid unit are collected by cameras deployed in various areas of the scenic area. A human recognition algorithm is used to study the spatiotemporal relationship of region G (i.e., the aforementioned target unit) within time period T, determining various spatiotemporal parameters of pedestrian flow, including: the flow of people within the region. (i.e., the aforementioned pedestrian flow), inter-regional pedestrian activity flow Crowd activity input stream Crowd activity output stream

[0040] In some embodiments of this application, determining the spatiotemporal parameters of pedestrian flow in a target unit within a grid cell during a first time period, based on the video image corresponding to the grid cell, includes the following steps: determining the spatiotemporal parameters of pedestrian flow in the target unit by identifying target human figures at various times in the video image. The spatiotemporal parameters of pedestrian flow include: pedestrian flow, crowd activity flow, crowd input flow, and crowd output flow. Pedestrian flow is used to characterize the number of people passing through the target unit during the first time period. Crowd activity flow is used to characterize the number of people transferring from the target unit to the other grid cells during the first time period. Crowd input flow is used to characterize the number of people flowing into the target unit during the first time period. Crowd output flow is used to characterize the number of people flowing from the target unit to the other grid cells during the first time period.

[0041] Specifically, assuming the grid area is divided into n grid cells G = {g1, g2, ..., gi, ..., gn}, and the time period is divided into m equal-step time intervals T = {t1, t2, ..., ti, ..., tm}, the population activity flow within the area... The number of people passing through the grid cell (target cell) g within a time interval t (equivalent to the first time interval mentioned above);

[0042] Crowd movement flow Let be the number of people who move from grid cell gi (i.e., the target cell mentioned above) to grid cell gj (i.e., all the other grid cells mentioned above) within a time interval t, where gi is the starting region of the flow and gj is the ending region of the flow, as shown below. Figure 3a As shown;

[0043] Crowd input stream The number of people flowing into the target cell gi from other grid cells within a time interval t;

[0044] Crowd output stream Let represent the number of people flowing from target cell gi to other grid cells within time interval t. The population input stream and population output stream are respectively as follows: Figure 3b , Figure 3c As shown.

[0045] In some embodiments of the present application, identifying the target human figure images at each moment in the video image includes the following steps: performing moving target detection on each frame image in the video image to obtain a moving target area; performing human figure contour recognition processing on the moving target area to obtain a human figure candidate area, wherein the human figure contour recognition processing at least includes: blob processing, area filtering, and aspect ratio filtering; using a human figure classifier model to classify and recognize the human figure candidate area to obtain a target human figure image, wherein the human figure classifier model is a model trained with preset sample data, and the preset sample data includes: first sample data that is determined to be a human figure candidate area through human figure contour recognition processing but is actually not a target human figure image, and second sample data that is determined to be a human figure candidate area through human figure contour recognition processing and is a target human figure image.

[0046] In order to improve the accuracy and effectiveness of the flow space-time parameters of the crowd, after obtaining the target human figure image, the method further includes the following steps: performing similarity matching between the target human figure image and a preset dressing image to obtain a target similarity; in the case where the target similarity is greater than a preset similarity threshold, determining the target human figure image as a staff image, and the staff image is not included in the crowd count during the process of determining the flow space-time parameters of the crowd.

[0047] Specifically, using the preset dressing image to perform matching recognition on the human figure image; when the preset dressing image is recognized in any current target human figure image, defining the current human figure image as a scenic area staff image, and in the solution of the present application, the scenic area staff do not participate in the calculation of the above-mentioned various flow space-time parameters of the crowd.

[0048] The present application uses surveillance video data to count the number of tourists, with high accuracy, and distinguishes the scenic area staff wearing the preset dressing to distinguish between the scenic area staff and tourists, avoiding including the scenic area staff in the statistics, and ensuring the high accuracy of the passenger flow statistics.

[0049] As an optional implementation manner, it is also possible to identify sudden situations in the scenic area according to real-time surveillance, such as equipment failures and tourist conflicts, and perform problem positioning and solution prediction, thereby improving the work efficiency of the scenic area service staff.

[0050] Step S206, using the target model to perform prediction based on the flow space-time parameters of the crowd to obtain the passenger flow of the target unit in the second time period, where the second time period is the next time period immediately following the first time period, and the target model is a model trained with historical space-time parameter data, and the historical space-time parameter data includes the flow space-time parameters of each grid unit in multiple historical time periods.

[0051] In some embodiments of this application, the prediction based on the spatiotemporal parameters of pedestrian flow using a target model includes the following steps: determining the grayscale value of the grid cell in the grid area based on the pedestrian flow to obtain a target flow map, wherein the grayscale value is used to characterize the pedestrian flow size corresponding to the grid cell; generating a target interaction flow map based on the crowd activity flow, crowd input flow, and crowd output flow; extracting the first spatial feature in the target flow map and the second spatial feature in the target interaction flow map through the convolutional neural network in the target model; extracting the target time feature corresponding to the first spatial feature and the second spatial feature in the first time period based on the gated loop control unit in the target model, and then fusing the first spatial feature, the second spatial feature, and the target time feature and inputting them into the fully connected layer of the target model for prediction.

[0052] Specifically, based on various spatiotemporal parameters of pedestrian flow at historical moments, a pedestrian flow map (i.e., the target flow map mentioned above) and a pedestrian interaction flow map (i.e., the target interaction flow map mentioned above) are generated, and the pedestrian flow map and the pedestrian interaction flow map are input into the neural network model for training.

[0053] Figure 4 This is a schematic diagram of the architecture of a target model provided according to an embodiment of this application, such as... Figure 4 As shown, the target model uses a local convolutional neural network to extract spatial features at different scales to describe spatial dependencies; and a gated recurrent unit (GRU) to extract temporal features to describe temporal dependencies. First, to enable the network to better learn spatial dependencies between locations, the network simultaneously considers static flow of population activity within a region and interactive flow of population between regions. Then, the spatial features extracted by the local convolutional neural network at different scales are fused. Next, the fused features are input into the gated recurrent unit. Time-series data not only has short-term dependencies but also a certain periodicity, i.e., long-term dependencies. Information transmission between units in the GRU enables the extraction and representation of both long-term and short-term features.

[0054] Specifically, crowd activities exhibit certain spatial correlations, and the crowd activities in a region are influenced by the spatial variables of its neighborhood. For the problem of predicting pedestrian flow, the flow at the next moment depends on the historical flow of the region. The flow of people between regions can strengthen the dynamic spatial relationship between regions. Based on this, this application uses a convolutional neural network to extract spatial features. The entire scenic area is converted into a regular grid by dividing it into regular grids. The flow within the grid is regarded as the gray level of the corresponding pixel, thereby converting it into an image. The crowd flow map and the crowd interaction flow map are divided into time intervals as network inputs. Static spatial features (i.e., the first spatial features mentioned above) are extracted through the crowd flow map, and dynamic spatial features (i.e., the second spatial features mentioned above) are extracted through the crowd interaction flow map.

[0055] To extract the spatial features of the target unit gi, for a time interval t, a crowd flow map is used. Interaction flow diagrams with localized groups As network input; assuming the image size corresponding to the target unit gi is r×r×1, then The image size is r×r×1. The image size is r×r×2, where 2 represents The system has two channels: one for the crowd input flow graph and one for the crowd output flow graph. Therefore, the output of the local convolutional neural network is as follows:

[0056]

[0057] Among them, symbols Represents tensor product; α v Represents static spatial features (i.e., the first spatial feature mentioned above); α f Represents dynamic spatial characteristics (i.e., the second spatial characteristic mentioned above); α r To represent the spatial features at a local image scale of r×r, the ReLU linear rectified function F() is used as the activation function. The weight vector of the convolutional layer; This is the bias term for the convolutional layer.

[0058] This application uses static crowd flow and dynamic crowd interaction flow as input to a convolutional neural network to extract spatial features. It also uses parallel convolution to fuse spatial multi-scale features and then uses a gated recurrent unit (GRU) to extract temporal features. This approach can improve prediction accuracy and learning efficiency, enabling crowd activity flow prediction and providing methodological support for perceiving the spatiotemporal movement patterns of humans.

[0059] In real life, tourist numbers surge during holidays, and the probability of unexpected situations at scenic spots also increases with severe weather. Therefore, in this application, to improve the accuracy of traffic flow prediction, the neural network model is trained not only based on the aforementioned spatiotemporal parameters of pedestrian flow from historical data, but also incorporates data on the season, holidays, and severe weather corresponding to these spatiotemporal parameters. Figure 5 As shown, this further improves the accuracy of pedestrian flow prediction.

[0060] As an optional implementation, the method further includes the following steps: obtaining weather forecast data corresponding to the second time period; determining a first identifier and a second identifier corresponding to the second time period, wherein the first identifier is used to characterize the season corresponding to the second time period, and the second identifier is used to characterize whether the second time period is a holiday; and using a target model, at least based on the weather forecast data, the first identifier, and the second identifier, predicting the flow of people in the target unit during the second time period.

[0061] In some embodiments of this application, after obtaining the flow of people in the target unit during the second time period, the method further includes the following steps: determining the estimated queuing time of the corresponding scenic spot of the target unit based on the flow of people in the target unit during the second time period, and sending the estimated queuing time to the display device in the target scenic area for display; and sending corresponding alarm prompt information when the flow of people exceeds a preset flow threshold.

[0062] Specifically, the current visitor flow data for each scenic area is input into a trained neural network model for prediction, obtaining information such as visitor flow and queuing time for each scenic area in the next moment. Based on the predicted visitor flow data, a visitor flow heat map of the entire scenic area is generated and sent to a large screen within the scenic area for display. This heat map is also synchronized to the scenic area management APP, allowing tourists to view the predicted visitor flow and queuing time for each scenic area in real time through the APP, and thus plan their travel routes.

[0063] If the predicted visitor flow exceeds the preset warning value or the queuing time at a certain attraction exceeds the preset duration, corresponding alarm information can be displayed on the scenic area's large screen and the scenic area management APP to remind tourists and staff.

[0064] Through the above steps, by deploying cameras at various scenic spots, and using human figure recognition and visitor flow prediction technologies, the distribution of visitor flow at each scenic spot is inferred. The real-time data is then displayed on the scenic area's large screen and management APP, achieving the goal of accurately predicting visitor flow at each scenic spot. This solves the technical problem that related technologies only analyze data from the scenic area's access control system when counting visitor flow, and cannot specifically predict the visitor flow at each scenic spot, resulting in low accuracy in visitor flow prediction and a poor visitor experience.

[0065] According to an embodiment of this application, an embodiment of a pedestrian flow prediction device is also provided. Figure 6 This is a schematic diagram of a pedestrian flow prediction device provided according to an embodiment of this application. Figure 6 As shown, the device includes:

[0066] The grid division module 60 is used to determine the grid area corresponding to the target scenic area. The grid area includes grid cells, and each grid cell corresponds to a scenic spot in the target scenic area.

[0067] The parameter determination module 62 is used to determine the spatiotemporal parameters of the pedestrian flow in the target unit of the grid cell within the first time period based on the video images corresponding to the grid cell. The spatiotemporal parameters of the pedestrian flow are used to characterize the pedestrian flow in the target unit within the first time period, as well as the pedestrian flow between the target unit and other grid cells. The target unit is the grid cell for which pedestrian flow prediction is planned.

[0068] The pedestrian flow prediction module 64 is used to predict the pedestrian flow of the target unit in the second time period based on the target model and the spatiotemporal parameters of pedestrian flow. The second time period is the time period immediately following the first time period. The target model is a model trained by historical spatiotemporal parameter data, which contains the spatiotemporal parameters of pedestrian flow of each grid unit in multiple historical time periods.

[0069] It should be noted that each module in the above-mentioned traffic flow prediction device can be a program module (for example, a set of program instructions to implement a certain function) or a hardware module. For the latter, it can be manifested in the following forms, but is not limited to them: each of the above modules is manifested as a processor, or the functions of each of the above modules are implemented by a processor.

[0070] It should be noted that the pedestrian flow prediction device provided in this embodiment can be used to perform... Figure 2 The pedestrian flow prediction method shown above is also applicable to the embodiments of this application, and will not be repeated here.

[0071] This application embodiment also provides a non-volatile storage medium, which includes a stored computer program. The device containing the non-volatile storage medium executes the following pedestrian flow prediction method by running the computer program: determining a grid area corresponding to a target scenic spot, wherein the grid area includes grid cells, each grid cell corresponding to a scenic spot in the target scenic spot; determining the spatiotemporal parameters of pedestrian flow in a target cell within a first time period based on video images corresponding to the grid cells, wherein the spatiotemporal parameters of pedestrian flow at least characterize the pedestrian flow of the target cell within the first time period, and the pedestrian flow between the target cell and other grid cells, the target cell being the grid cell for which pedestrian flow prediction is planned; and using a target model to predict the pedestrian flow of the target cell within a second time period based on the spatiotemporal parameters of pedestrian flow, wherein the second time period is the time period immediately following the first time period, the target model being a model trained using historical spatiotemporal parameter data, the historical spatiotemporal parameter data containing the spatiotemporal parameters of pedestrian flow for each grid cell in multiple historical time periods.

[0072] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0073] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0074] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0075] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0076] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0077] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0078] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for predicting pedestrian flow, characterized in that, include: Determine the grid area corresponding to the target scenic area, wherein the grid area includes grid cells, and each grid cell corresponds to a scenic spot in the target scenic area; Based on the video images corresponding to the grid unit, the spatiotemporal parameters of the pedestrian flow in the target unit of the grid unit within the first time period are determined. The spatiotemporal parameters of the pedestrian flow are used to characterize the pedestrian flow of the target unit within the first time period and the pedestrian flow between the target unit and the other grid units. The target unit is the grid unit for which pedestrian flow prediction is planned. A target model is used to predict the pedestrian flow within a second time period based on the aforementioned spatiotemporal parameters. This prediction involves determining the grayscale value of each grid cell within the grid area based on the pedestrian flow, thus obtaining a target flow map. A target interaction flow map is generated based on pedestrian activity flow, pedestrian input flow, and pedestrian output flow. This target interaction flow map is used to extract second spatial features, where the grayscale value represents the pedestrian flow magnitude corresponding to the grid cell. The target flow map is used to extract first spatial features, thereby predicting the pedestrian flow within the second time period. The second time period is the period immediately following the first time period. The target model is trained using historical spatiotemporal parameter data, which includes the pedestrian flow spatiotemporal parameters of each grid cell across multiple historical time periods. For a given time interval t, the target flow map is used... Interaction flow graph with the target As input to the convolutional neural network; the image size corresponding to the target unit is r×r×1, the image size of the target flow graph is r×r×1, and the image size of the target interaction flow graph is r×r×2. The target interaction flow graph includes two channels: one channel for the crowd input flow graph and one channel for the crowd output flow graph. The output of the local convolutional neural network is shown below: ； in, Represents tensor product, This is the first spatial feature. This is the second spatial feature. For spatial features on a local image scale of r×r, the linear rectified function ReLU is used as the activation function. The weight vector of the convolutional layer. This refers to the bias term of the convolutional layer; Based on the gated loop control unit in the target model, the target temporal features corresponding to the spatial features on the local image scale of r×r are extracted, and the target temporal features are input to the next gated loop control unit. Finally, the input is input to the fully connected layer of the target model for prediction. The input of each gated loop control unit is the spatial features on the local image of a time interval.

2. The method for predicting pedestrian flow according to claim 1, characterized in that, Based on the video images corresponding to the grid cells, the spatiotemporal parameters of pedestrian flow in the target cells within the first time period are determined as follows: By identifying target human figures in the video image at various times, the spatiotemporal parameters of the pedestrian flow in the target unit are determined. These parameters include pedestrian flow, crowd activity flow, crowd input flow, and crowd output flow. The pedestrian flow represents the number of people passing through the target unit during the first time period. The crowd activity flow represents the number of people moving from the target unit to other grid units during the first time period. The crowd input flow represents the number of people flowing into the target unit during the first time period. The crowd output flow represents the number of people flowing from the target unit to other grid units during the first time period.

3. The method for predicting pedestrian flow according to claim 2, characterized in that, Using the target model, prediction based on the aforementioned spatiotemporal parameters of pedestrian flow includes: The first spatial feature in the target flow graph and the second spatial feature in the target interaction flow graph are extracted using the convolutional neural network in the target model.

4. The method for predicting pedestrian flow according to claim 2, characterized in that, Identifying target human figures at various times in the video footage includes: Motion target detection is performed on each frame of the video image to obtain the motion target region; The moving target region is subjected to human contour recognition processing to obtain human candidate regions, wherein the human contour recognition processing includes at least: block processing, area filtering and aspect ratio filtering; A human figure classifier model is used to classify and identify the human figure candidate region to obtain the target human figure image. The human figure classifier model is a model trained with preset sample data. The preset sample data includes: first sample data that is determined to be a human figure candidate region after the human figure contour recognition processing, but is not actually the target human figure image, and second sample data that is determined to be a human figure candidate region after the human figure contour recognition processing and is the target human figure image.

5. The pedestrian flow prediction method according to claim 4, characterized in that, After obtaining the target human-shaped image, the method further includes: The target human figure image is matched with a preset clothing image to obtain the target similarity. If the target similarity is greater than a preset similarity threshold, the target human image is identified as a staff member image, wherein the staff member image is not included in the number of people in the process of determining the spatiotemporal parameters of the crowd.

6. The method for predicting pedestrian flow according to claim 1, characterized in that, The method further includes: Obtain the weather forecast data corresponding to the second time period; Determine a first identifier and a second identifier corresponding to the second time period, wherein the first identifier is used to characterize the season corresponding to the second time period, and the second identifier is used to characterize whether the second time period is a holiday; Using the target model, at least based on the weather forecast data, the first identifier, and the second identifier, the pedestrian flow of the target unit during the second time period is predicted.

7. The method for predicting pedestrian flow according to claim 1, characterized in that, After obtaining the pedestrian flow of the target unit during the second time period, the method further includes: Based on the visitor flow of the target unit during the second time period, the estimated queuing time for the corresponding attraction of the target unit is determined, and the estimated queuing time is sent to the display devices within the target scenic area for display; and, If the number of people exceeds a preset threshold, a corresponding alarm message will be sent.

8. A pedestrian flow prediction device, characterized in that, include: A grid division module is used to determine the grid area corresponding to the target scenic area, wherein the grid area includes grid cells, and each grid cell corresponds to a scenic spot in the target scenic area; The parameter determination module is used to determine the spatiotemporal parameters of the pedestrian flow of the target unit in the grid unit within a first time period based on the video images corresponding to the grid unit. The spatiotemporal parameters of the pedestrian flow are used to characterize the pedestrian flow of the target unit within the first time period and the pedestrian flow between the target unit and the other grid units. The target unit is the grid unit for which pedestrian flow prediction is planned. The pedestrian flow prediction module is used to predict the pedestrian flow of the target unit within a second time period using a target model based on the aforementioned spatiotemporal parameters. Specifically, based on the pedestrian flow, the grayscale value of each grid cell in the grid area is determined to obtain a target flow map. A target interaction flow map is generated based on crowd activity flow, crowd input flow, and crowd output flow. This target interaction flow map is used to extract second spatial features, where the grayscale value represents the pedestrian flow magnitude corresponding to the grid cell. The target flow map is used to extract first spatial features, thereby predicting the pedestrian flow within the second time period. The second time period is the period immediately following the first time period. The target model is a model trained using historical spatiotemporal parameter data, which includes the pedestrian spatiotemporal parameters of each grid cell across multiple historical time periods. For a time interval t, the target flow map is used... Interaction flow graph with the target As input to the convolutional neural network; the image size corresponding to the target unit is r×r×1, the image size of the target flow graph is r×r×1, and the image size of the target interaction flow graph is r×r×2. The target interaction flow graph includes two channels: one channel for the crowd input flow graph and one channel for the crowd output flow graph. The output of the local convolutional neural network is shown below: ； in, Represents tensor product, This is the first spatial feature. This is the second spatial feature. For spatial features on a local image scale of r×r, the linear rectified function ReLU is used as the activation function. The weight vector of the convolutional layer. This refers to the bias term of the convolutional layer; Based on the gated loop control unit in the target model, the target temporal features corresponding to the spatial features on the local image scale of r×r are extracted, and the target temporal features are input to the next gated loop control unit. Finally, the input is input to the fully connected layer of the target model for prediction. The input of each gated loop control unit is the spatial features on the local image of a time interval.

9. An electronic device, characterized in that, include: A memory and a processor, the processor being configured to run a program stored in the memory, wherein the program, when running, executes the pedestrian flow prediction method according to any one of claims 1 to 7.

10. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored computer program, wherein the device containing the non-volatile storage medium executes the pedestrian flow prediction method according to any one of claims 1 to 7 by running the computer program.

Images