Pedestrian Trajectory Estimation Method, Device, Equipment and Medium Based on Crowd Heat Map

Abstract

The present invention discloses a pedestrian trajectory estimation method based on a crowd heat map, comprising the steps of: obtaining a spatial topology map of the camera layout, and determining the starting and ending camera points and time of the pedestrians to be recognized; obtaining the crowd heat map of each camera in the spatial topology map within the starting and ending camera points and time of the pedestrians to be recognized; obtaining the most probable path of pedestrians passing between any camera and its adjacent camera according to the crowd heat map of each camera; and obtaining the estimated action trajectory of the pedestrians at the starting and ending positions according to the most probable path of pedestrians passing between any camera and its adjacent camera. This application can solve the problem of the failure of pedestrian trajectory tracking at some points in the topology map caused by detection errors due to factors such as light, clothing, and wearing masks. For abnormal behavior personnel who do not travel according to the estimated action trajectory, early warning can be achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of trajectory estimation, and in particular to a pedestrian trajectory estimation method, device, electronic device and storage medium based on a crowd heat map. Background Art

[0002] Pedestrian trajectory tracking is an important branch of object tracking, aiming to obtain the activity trajectory of a specific pedestrian within a certain time and area. Based on the trajectory, the preferences, intentions, etc. of the specific pedestrian can be judged.

[0003] In the prior art, face recognition methods are mainly used for tracking pedestrian trajectories. Such methods first need to implement the recognition of face images, and then use tracking algorithms, etc. to track pedestrian trajectories, and then determine the pedestrian trajectories based on the tracking results of multiple cameras. However, these methods are easily affected by adverse factors such as light, occlusion, and small image resolution. Especially in some monitoring scenarios, it is difficult to guarantee the accuracy of face recognition when wearing a mask, and algorithms for recognition based on features such as clothing and body posture are also easily affected by factors such as changing clothes and changes in body posture caused by angle transformation, and it is impossible to ensure that the same pedestrian is accurately recognized and tracked by all cameras he passes by, resulting in incorrect feature matching or the inability to find a matching portrait in other cameras, and thus the portrait trajectory is incorrect or discontinuous in the regional topology map.

[0004] Therefore, there is a need for a method that can analyze the most probable path of a pedestrian passing through a camera according to the crowd heat map situation of each camera, and then estimate the action trajectory of the starting and ending positions of the pedestrian. Summary of the Invention

[0005] To this end, the present invention provides a pedestrian trajectory estimation method, device, electronic device and storage medium based on a crowd heat map, in an attempt to solve or at least alleviate at least one of the above problems.

[0006] According to an aspect of the present invention, a pedestrian trajectory estimation method based on a population heat map is provided. The method estimates the pedestrian trajectory by obtaining the heat map of the personnel distribution under the camera and calculating the weight of the path passed by the pedestrian between adjacent cameras. The method includes the steps of: obtaining the spatial topology map of the camera layout, and determining the starting and ending camera points and time of the pedestrians to be recognized; according to the spatial topology map of the camera layout and the starting and ending camera points and time of the pedestrians to be recognized, obtaining the population heat map of each camera in the spatial topology map within the starting and ending camera points and time of the pedestrians to be recognized; according to the population heat map of each camera, obtaining the most probable path passed by the pedestrians between any camera and its adjacent camera; and according to the most probable path passed by the pedestrians between any camera and its adjacent camera, obtaining the estimated movement trajectory of the pedestrians at the starting and ending positions.

[0007] Optionally, in the pedestrian trajectory estimation method based on a population heat map according to the present invention, the step of obtaining the spatial topology map of the camera layout and determining the starting and ending camera points of the pedestrians to be recognized includes: obtaining the camera information and road information within the set area; according to the camera information and road information within the set area, obtaining the adjacent information of the cameras within the set area; according to the adjacent information of the cameras within the set area, obtaining the spatial topology map of the camera layout; and according to the spatial topology map of the camera layout, obtaining the starting and ending camera points of the pedestrians to be recognized, and the position information of the camera points in the spatial topology map.

[0008] Optionally, in the pedestrian trajectory estimation method based on a population heat map according to the present invention, the step of obtaining the population heat map of each camera in the spatial topology map within the starting and ending camera points and time of the pedestrians to be recognized according to the spatial topology map of the camera layout and the starting and ending camera points and time of the pedestrians to be recognized includes: obtaining the pedestrian quantity information collected by each camera in the spatial topology map within the set time interval; according to the pedestrian quantity information collected by each camera in the spatial topology map within the set time interval, obtaining the population heat map of each camera in the spatial topology map; and according to the population heat map of each camera in the spatial topology map and the starting and ending camera points and time of the pedestrians to be recognized, obtaining the cameras passed by when the pedestrians walk at the starting and ending camera points to be recognized, and the population heat map of each camera within the time to be recognized.

[0009] Optionally, in the pedestrian trajectory estimation method based on crowd heatmaps according to the present invention, the step of obtaining the most probable path of pedestrians passing between any camera and its adjacent camera based on the crowd heatmaps of each camera includes: obtaining the crowd heatmaps of any two adjacent cameras on the same road based on the crowd heatmaps of each camera and road information; obtaining the weights of pedestrians passing between two adjacent cameras on the same road based on the crowd heatmaps of any two adjacent cameras on the same road; and obtaining the most probable path of pedestrians passing between any camera and its adjacent camera based on the weights of pedestrians passing between two adjacent cameras on the same road.

[0010] Optionally, in the pedestrian trajectory estimation method based on crowd heatmaps according to the present invention, the step of obtaining the weights of pedestrians passing between two adjacent cameras on the same road based on the crowd heatmaps of any two adjacent cameras on the same road includes: comparing the crowd heatmaps of any two adjacent cameras on the same road; if the crowd heatmaps of any two adjacent cameras on the same road are the same, the weight of pedestrians passing between the two adjacent cameras on the same road is 1; if the crowd heatmaps of any two adjacent cameras on the same road are different, obtaining the crowd heatmap information of one of the cameras and all its adjacent cameras; and based on the weights of pedestrians passing between one of the cameras and all its adjacent cameras, the sum of the weights of pedestrians passing between the camera with a larger heatmap value and all its adjacent cameras is 1.

[0011] Optionally, in the pedestrian trajectory estimation method based on crowd heatmaps according to the present invention, the step of obtaining the estimated action trajectory of a pedestrian at the starting and ending positions based on the most probable path of pedestrians passing between any camera and its adjacent camera includes: obtaining the most probable path of pedestrians passing through the camera at the starting position of the pedestrian and predicting the next camera that the pedestrian will pass through based on the most probable path of pedestrians passing between any camera and its adjacent camera and the starting and ending camera points of the pedestrian to be identified; predicting each camera that the pedestrian will pass through in sequence based on the most probable path of pedestrians passing between any camera and its adjacent camera; and obtaining the estimated action trajectory of the pedestrian at the starting and ending positions based on each predicted camera that the pedestrian passes through.

[0012] Optionally, in the method for estimating pedestrian trajectories based on a crowd heat map according to the present invention, the steps of the method further include: obtaining distance information between adjacent cameras among the cameras through which the estimated movement trajectory of a pedestrian passes at the starting and ending positions; obtaining arrival time information of each camera through which the estimated movement trajectory of the pedestrian passes at the starting and ending positions based on the distance information between adjacent cameras and the traveling speed information of the pedestrian; and obtaining abnormal behavior information of the pedestrian based on the arrival time information of each camera through which the estimated movement trajectory of the pedestrian passes at the starting and ending positions and the actual arrival information of the pedestrian at the corresponding camera.

[0013] According to another aspect of the present invention, there is provided an apparatus for estimating pedestrian trajectories based on a crowd heat map. The apparatus estimates pedestrian trajectories by obtaining a heat map of the distribution of people under cameras and calculating the weights of the paths between adjacent cameras passed by pedestrians. The apparatus includes: an obtaining module, configured to obtain a spatial topology map of the camera layout and determine the starting and ending camera points and times of the pedestrians to be identified; a heat map calculation module, configured to obtain a crowd heat map of each camera in the spatial topology map within the starting and ending camera points and times of the pedestrians to be identified based on the spatial topology map of the camera layout and the starting and ending camera points and times of the pedestrians to be identified; a path calculation module, configured to obtain the most probable path passed by pedestrians between any camera and its adjacent camera based on the crowd heat map of each camera; and obtain the estimated movement trajectory of the pedestrian at the starting and ending positions based on the most probable path passed by pedestrians between any camera and its adjacent camera.

[0014] According to another aspect of the present invention, there is provided a computing device, including: one or more processors; and a memory; one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any of the methods in the method for estimating pedestrian trajectories based on a crowd heat map as described above.

[0015] According to another aspect of the present invention, there is provided a computer-readable storage medium storing one or more programs, the one or more programs including instructions that, when executed by a computing device, cause the computing device to execute any of the methods in the method for estimating pedestrian trajectories based on a crowd heat map as described above.

[0016] According to the solution for pedestrian trajectory estimation based on crowd heatmap of the present invention, by obtaining the spatial topology map of the camera layout, the starting and ending camera points and time of the pedestrians to be recognized are determined; according to the spatial topology map of the camera layout and the starting and ending camera points and time of the pedestrians to be recognized, the crowd heatmap of each camera in the spatial topology map is obtained within the starting and ending camera points and time of the pedestrians to be recognized; according to the crowd heatmap of each camera, the maximum probability path of pedestrians passing between any camera and its adjacent camera is obtained; according to the maximum probability path of pedestrians passing between any camera and its adjacent camera, the estimated action trajectory of the pedestrians at the starting and ending positions is obtained. Through the crowd heatmap method of the camera, the maximum probability path between each camera and its adjacent camera of the pedestrians between the starting and ending positions is obtained, and then the estimated action trajectory of the pedestrians is obtained, which can solve the problem of the failure of pedestrian trajectory tracking at some points in the topology map caused by detection errors due to factors such as light, clothing, and wearing masks. For abnormal behavior personnel who do not travel according to the estimated action trajectory, early warning can be achieved to prevent dangerous events from occurring. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] To achieve the above and related purposes, certain illustrative aspects are described herein in connection with the following description and the accompanying drawings, which indicate various ways in which the principles disclosed herein can be practiced, and all aspects and their equivalent aspects are intended to fall within the scope of the claimed subject matter. The above and other objects, features, and advantages of the present disclosure will become more apparent by reading the following detailed description in conjunction with the accompanying drawings. Throughout the present disclosure, like reference numerals generally refer to like components or elements.

[0018] Figure 1 FIG. shows a schematic structural diagram of a computing device 100 according to an embodiment of the present invention; and

[0019] Figure 2 FIG. shows a flowchart of a method 200 for pedestrian trajectory estimation based on crowd heatmap according to an embodiment of the present invention; and

[0020] Figure 3 FIG. shows a schematic structural diagram of a device 300 for pedestrian trajectory estimation based on crowd heatmap according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be completely conveyed to those skilled in the art.

[0022] Figure 1 is a block diagram of an example computing device 100. In a basic configuration 102, the computing device 100 typically includes a system memory 106 and one or more processors 104. A memory bus 108 can be used for communication between the processor 104 and the system memory 106.

[0023] Depending on the desired configuration, the processor 104 can be any type of processing, including but not limited to: a microprocessor (μP), a microcontroller (μC), a digital signal processor (DSP), or any combination thereof. The processor 104 can include one or more levels of cache, such as a level 1 cache 110 and a level 2 cache 112, a processor core 114, and registers 116. An example processor core 114 can include an arithmetic logic unit (ALU), a floating point unit (FPU), a digital signal processing core (DSP core), or any combination thereof. An example memory controller 118 can be used with the processor 104, or in some implementations, the memory controller 118 can be an internal part of the processor 104.

[0024] Depending on the desired configuration, the system memory 106 can be any type of memory, including but not limited to: volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.), or any combination thereof. The system memory 106 can include an operating system 120, one or more applications 122, and program data 124. In some embodiments, the applications 122 can be arranged to operate on the operating system using the program data 124. In some embodiments, the computing device 100 is configured to execute a pedestrian trajectory estimation method 200 based on a crowd heat map, which estimates the pedestrian trajectory by obtaining the heat map of the personnel distribution under the camera and calculating the weights of the paths between adjacent cameras. The program data 124 contains instructions for executing the method 200.

[0025] The computing device 100 may also include an interface bus 140 that facilitates communication from various interface devices (e.g., output device 142, peripheral interface 144, and communication device 146) to the basic configuration 102 via the bus / interface controller 130. Example output devices 142 include a graphics processing unit 148 and an audio processing unit 150. They may be configured to facilitate communication with various external devices such as a display or speakers via one or more A / V ports 152. Example peripheral interfaces 144 may include a serial interface controller 154 and a parallel interface controller 156, which may be configured to facilitate communication with external devices such as input devices (e.g., keyboard, mouse, pen, voice input device, touch input device) or other peripherals (e.g., printer, scanner, etc.) via one or more I / O ports 158. Example communication device 146 may include a network controller 160, which may be arranged to facilitate communication with one or more other computing devices 162 via one or more communication ports 164 through a network communication link. In this embodiment, a spatial topology map of the camera layout can be obtained by a data acquisition device, and the starting and ending camera points and time of the pedestrians to be recognized can be determined.

[0026] The network communication link can be an example of a communication medium. The communication medium can generally embody computer-readable instructions, data structures, program modules in a modulated data signal such as a carrier wave or other transmission mechanism, and can include any information delivery medium. A "modulated data signal" can be a signal in which one or more of its data concentrations or its changes can encode information in the signal. As a non-limiting example, the communication medium can include wired media such as a wired network or a dedicated line network, and various wireless media such as sound, radio frequency (RF), microwave, infrared (IR), or other wireless media. The term computer-readable medium used herein can include both storage media and communication media. In some embodiments, one or more programs are stored in the computer-readable medium, and the one or more programs include instructions for performing certain methods. For example, according to an embodiment of the present invention, the computing device 100 executes the pedestrian trajectory estimation method 200 based on the crowd heat map through the instructions.

[0027] The computing device 100 can be implemented as part of a small-sized portable (or mobile) electronic device, which can be such as a cellular phone, a personal digital assistant (PDA), a personal media player device, a wireless network browsing device, a personal head-mounted device, an application-specific device, or a hybrid device that can include any of the above functions. The computing device 100 can also be implemented as a personal computer including a desktop computer and a laptop computer configuration.

[0028] Figure 2The flowchart of a pedestrian trajectory estimation method 200 based on a population heat map according to an embodiment of the present invention is shown. As Figure 2 shown, the method 200 estimates the pedestrian trajectory by obtaining the heat map of the personnel distribution under the camera and calculating the weight of the path between adjacent cameras. The method 200 starts from step S210, obtains the spatial topology map of the camera layout, and determines the starting and ending camera points and time of the pedestrians to be identified.

[0029] Specifically, in a city, cameras are generally installed at road intersections, some important positions in the middle of the road, etc. In the embodiments of the present application, a camera installed at a position is regarded as a camera point. For example, there may be multiple cameras at a road intersection, such as at the four corners of the intersection, and multiple cameras are installed in multiple directions and multiple lanes at each corner. In this embodiment, all the cameras installed at this intersection are regarded as a camera point, and the number of people obtained by all the cameras within the set time is the number of personnel obtained by this camera point. In practical applications, there may be the following scenarios. For example, one camera is installed at each of the four corners of a road intersection. If a person goes straight from one direction to another direction, it is possible that the pedestrian enters the intersection and is captured by one camera, and then is captured by another camera when leaving the intersection. At this time, the number of people obtained by this camera point has double repetition. Regarding the method of collecting the number of personnel by the camera point, the present application does not discuss it in detail.

[0030] Specifically, it can be known that the paths of pedestrians are all along the road. If a pedestrian appears at the camera installed at the road entrance, then the pedestrian will definitely appear at the camera at the exit of this road. For the camera points at intersections, the paths of their actions include four directions, so there are also four paths for them to walk. Since the destinations reached in each direction are different, therefore, among these paths, the number of people from the camera points at intersections to the cameras in the four adjacent directions is also different. Then, by calculating the number of people reaching the four directions within a certain time, the personnel distribution from this camera point to the four directions can be obtained.

[0031] Specifically, in the specific implementation, by obtaining the distribution of cameras within a certain area, and then according to the road distribution situation within this area, the spatial topology map of these camera distributions can be obtained. At this time, if the camera point of the starting position of the pedestrian and the camera point of the ending position are obtained, the cameras that the pedestrian can pass through on the walking path from the starting position to the ending position can be obtained through the spatial topology map of the camera distribution.

[0032] Specifically, in an embodiment of the present application, the steps of obtaining the spatial topology map of the camera layout and determining the starting and ending camera points of the pedestrians to be recognized include:

[0033] Obtain the camera information and road information within the set area; specifically, select a certain area, obtain the roads within the area, and all the cameras set on the roads. The road information is the path that pedestrians need to pass through, and the cameras on the road can obtain the information of pedestrians passing by when pedestrians pass through the road where the camera is located. For example, the camera can count the number of pedestrians passing by.

[0034] According to the camera information and road information within the set area, obtain the adjacent information of the cameras within the set area; specifically, the adjacent information of the cameras is the physical adjacency relationship. For example, on a road, when a pedestrian walks along the road and passes a camera, the next camera passed is the adjacent camera of the previous camera. Therefore, it can be known that on a road, a camera has a previous and a next adjacent camera. However, for the cameras at road intersections, it has multiple adjacent cameras. The specific number of adjacent cameras is related to the number of passing roads at the intersection. There is one adjacent camera for each passing direction. For example, the camera point at a crossroads has four adjacent cameras, that is, when a pedestrian walks at the position of the crossroads, he will be photographed by one of the four adjacent cameras.

[0035] According to the adjacent information of the cameras within the set area, obtain the spatial topology map of the camera layout; specifically, after obtaining the adjacent information of the cameras within the area, the spatial topology map of the cameras can be planned according to the adjacent relationship of the cameras. Through the spatial topology map of the camera layout, all the routing situations that can be passed from any one camera to a destination camera can be obtained.

[0036] According to the spatial topology map of the camera layout, obtain the starting and ending camera points of the pedestrians to be recognized, and the position information of the camera points in the spatial topology map. Specifically, after obtaining the spatial topology map of the camera layout, according to the spatial position of each camera in the spatial topology map, combined with the starting position and the ending position of the pedestrian, the camera points at the starting position and the ending position of the pedestrian can be obtained, and the walking path of the pedestrian can be planned.

[0037] Through step S220, according to the spatial topology map of the camera layout, the starting and ending camera points of the pedestrians to be recognized, and the time, obtain the crowd heat map of each camera in the spatial topology map within the starting and ending camera points of the pedestrians to be recognized and the time.

[0038] Specifically, the crowd heat map refers to the number of people passing by each camera within a unit time. Therefore, it can be known that on the same road, the number of people passing by each camera is the same. However, for the cameras set at road intersections, it is different. Its crowd heat map reflects the probability of the roads that pedestrians take.

[0039] Specifically, in the embodiment of the present application, the step of obtaining the crowd heat map of each camera in the spatial topology map at the starting and ending camera points and time of the pedestrians to be recognized according to the spatial topology map based on the camera layout and the starting and ending camera points and time of the pedestrians to be recognized includes:

[0040] Obtain the pedestrian quantity information collected by each camera in the spatial topology map within a set time interval; specifically, the set time interval should have a certain degree of typicality and cannot be some special dates or special seasons, etc. The pedestrian quantity information collected by each camera is the information of pedestrians passing by the camera once. Specifically, if a pedestrian lingers repeatedly or leaves and returns multiple times within the area of the camera and is not collected by other cameras, then only count the pedestrian passing by the camera once because the action trajectory of the pedestrian does not deviate from the statistical interval of the camera and does not complete the departure from the collection area of the camera and being collected by other cameras, and does not complete a walking path.

[0041] Obtain the crowd heat map of each camera in the spatial topology map according to the pedestrian quantity information collected by each camera in the spatial topology map within the set time interval; the crowd heat map refers to the number of pedestrians within a unit time. Therefore, after collecting the pedestrian quantity information of each camera within the set time length, the number of pedestrians can be averaged to obtain the number of pedestrians within a unit time. Specifically, the crowd heat map has a great correlation with time. For example, within a day, the number of people during the day is more than that at night, and the number of people during the morning and evening rush hours is more than that during ordinary periods. Therefore, when setting the time interval for the camera to collect the number of pedestrians, if the set time span is very large, it is necessary to divide the collected number of people into crowd heat maps according to time periods. For example, one hour can be used as a time period to obtain the crowd heat map of each camera for each hour. Such a crowd heat map is more practical. A large parameter of the crowd heat map indicates a large number of people passing by the camera, and a small parameter of the crowd heat map indicates a small number of people passing by the camera.

[0042] Based on the crowd heat map of each camera in the spatial topology map, the starting and ending camera positions and time of the pedestrians to be recognized, obtain the cameras passed by when the pedestrians to be recognized walk at the starting and ending camera positions, and the crowd heat map of each camera within the time to be recognized. Specifically, after obtaining the crowd heat map of each camera in the spatial topology map, the crowd heat map situation of all camera positions passed during the walking period of the pedestrians can be obtained according to the starting position camera and the ending position camera of the pedestrians.

[0043] Through step S230, based on the crowd heat map of each camera, obtain the most probable path of pedestrians passing between any camera and its adjacent camera.

[0044] Specifically, since the crowd heat map of each camera is different, the probability of a pedestrian walking from one camera to its adjacent camera is also different. For example, for a camera at a crossroads, it has four adjacent cameras, and the positions of the four adjacent cameras are the railway station, the park, a small residential area, and a small restaurant respectively. When the crowd heat map of the camera at the crossroads and the crowd heat maps of its four adjacent cameras are known, for example, the crowd heat map of the railway station is greater than that of the park, which is greater than that of the residential area, which is greater than that of the small restaurant. Therefore, it can be predicted that the probability of a pedestrian going from this crossroads to the railway station is greater than that to the park, which is greater than that to the small residential area, which is greater than that to the small restaurant. That is, the most probable path of the pedestrian is from the camera at this crossroads to the camera at the railway station. By obtaining the most probable path, it is possible to know the position where the pedestrian is most likely to have passed and also to judge whether the walking trajectory of the pedestrian is normal.

[0045] Specifically, in the embodiment of the present application, the step of obtaining the most probable path of pedestrians passing between any camera and its adjacent camera based on the crowd heat map of each camera includes:

[0046] Based on the crowd heat map of each camera and the road information, obtain the crowd heat maps of any two adjacent cameras on the same road; specifically, obtaining the crowd heat maps of two adjacent cameras is to count the crowd heat map situation on the road between these two cameras. It can be judged that if the crowd heat maps of two cameras are higher than those of other cameras, then it is highly probable that pedestrians pass through the road with a high crowd heat map.

[0047] Based on the crowd heat maps of any two adjacent cameras on the same road, obtain the weight of pedestrians passing between two adjacent cameras on the same road. Specifically, the method of calculating the weight of pedestrians passing between adjacent cameras can use the weighted average method. For example, set the weight of the local camera to 0.6 and the weight of its adjacent camera to 0.4. Suppose the crowd heat map of the camera at a crossroads from 11:00 to 12:00 noon is 2000, and the crowd heat maps of the four adjacent cameras at this time are 3000, 1000, 600, and 200 respectively. Then, the weights of the pedestrian walking paths from the camera at the crossroads to the four adjacent cameras can be calculated as 2800, 1600, 1440, and 1280 respectively, and then obtain the probabilities of pedestrians reaching the other four cameras from the camera, which are: 0.40, 0.22, 0.20, and 0.18. The weight of pedestrians passing between two adjacent cameras is also the heat map situation of the road.

[0048] Based on the weight of pedestrians passing between two adjacent cameras on the same road, obtain the maximum probability path of pedestrians passing between any camera and its adjacent camera. Specifically, after calculating the weight of pedestrians passing between two adjacent cameras, the maximum probability path of pedestrians passing between the two cameras with the largest weight can be obtained. For example, for the road with the weight of the pedestrian walking path of 2800 mentioned above, the maximum probability of pedestrians passing through it is 0.4, and this path is the maximum probability path of pedestrians passing through.

[0049] Specifically, in the embodiment of the present application, the step of obtaining the weight of pedestrians passing between two adjacent cameras on the same road based on the crowd heat maps of any two adjacent cameras on the same road includes:

[0050] Compare the crowd heat maps of any two adjacent cameras on the same road. Specifically, comparing the crowd heat maps of a camera and its adjacent camera is to determine whether the two adjacent cameras are set at the intersection of the road or in the middle of the road. It can be known that if the cameras are set in the middle of the road, then the probability of pedestrians walking between these two cameras is 1, that is, they will definitely pass through these two cameras, and the number of people passing through these two cameras is exactly the same. For example, cameras A and B are set in the middle of a certain road, and neither of these two cameras is at the road intersection, that is, the pedestrians of camera A must go to camera B, and similarly, the pedestrians of camera B must go to camera A. If the crowd heat maps of adjacent cameras are different, then it can be determined that one of the cameras is located at the road intersection, because the cameras at the road intersection can collect the people coming from other roads, while the cameras in the middle of the road can only collect part of the people diverted from the intersection cameras.

[0051] If the crowd heat maps of any two adjacent cameras on the same road are the same, the weight of pedestrians passing between the two adjacent cameras on the same road is 1; specifically, when the crowd heat maps of any two adjacent cameras on the same road are the same, it can be determined that both of these two cameras are in the middle of the road, or the cameras at an intersection only collect the personnel on one road, and there is no personnel diversion on other branch roads. For example, cameras A and B are set in the middle of a road, then the probability of pedestrians walking from camera A to camera B and the probability of pedestrians walking from camera B to camera A are both 1.

[0052] If the crowd heat maps of any two adjacent cameras on the same road are different, obtain the crowd heat map information of one of the cameras and all its adjacent cameras; specifically, when the crowd heat maps are different, it can be known that these two cameras must not be the cameras set in the middle of the road, then there must be at least one camera set at the intersection of the road, that is, at the starting point or the end point of the road. At this time, by comparing the number of adjacent cameras, it can be judged whether these two cameras are the cameras at the intersection of the road, or one is the camera at the middle position and the other is the camera at the intersection of the road. Finally, obtain the crowd heat maps of the cameras at the intersection of the road and their adjacent cameras.

[0053] Based on the weight of pedestrians passing between one of the cameras and all its adjacent cameras, the sum of the weights of pedestrians passing between the camera with the larger heat map value and all its adjacent cameras is 1. Specifically, after obtaining the camera at the intersection of the road, according to the number of roads that the intersection of the road can lead to, determine how many adjacent cameras this camera has, and then obtain the crowd heat maps of all adjacent cameras. According to the calculation method of the weight of pedestrians passing between adjacent cameras, the probability of pedestrians passing through each road can be obtained, and the sum of these probabilities is 1, that is, among the crowd numbers collected by the camera at the intersection of the road, the sum of the numbers of personnel who will surely walk towards each road that the intersection of the road leads to. For example, the camera at the intersection of the road collects 4000 people, then these 4000 people will surely be diverted through the roads branched from the intersection of the road, and pedestrians cannot be at the intersection of the road and never move.

[0054] Through step S240, based on the maximum probability path of pedestrians passing between any camera and its adjacent cameras, obtain the estimated movement trajectory of pedestrians at the starting and ending positions.

[0055] Specifically, after obtaining the most probable path passed by the pedestrian, the action trajectory of the pedestrian can be estimated based on the starting position and the ending position of the pedestrian. The action trajectory of the pedestrian reaches the next camera according to the most probable path after passing each camera point.

[0056] Specifically, in the embodiment of the present application, the step of obtaining the estimated action trajectory of the pedestrian at the starting and ending positions according to the most probable path passed by the pedestrian between any camera and the adjacent camera includes:

[0057] According to the most probable path passed by the pedestrian between any camera and the adjacent camera, and the starting and ending camera points of the pedestrian to be recognized, obtain the most probable path passed by the pedestrian at the starting position camera of the pedestrian, and predict the next camera passed by the pedestrian; specifically, after obtaining the most probable path passed by the pedestrian between each camera and its adjacent camera, it is possible to judge the point of the next camera that the pedestrian most probably reaches at the current camera point according to the most probable path.

[0058] According to the most probable path passed by the pedestrian between any camera and the adjacent camera, sequentially predict each camera passed by the pedestrian;

[0059] According to each camera passed by the predicted pedestrian, obtain the estimated action trajectory of the pedestrian at the starting and ending positions.

[0060] Specifically, the estimated action trajectory of the pedestrian is also related to the path length between the cameras. For example, when a pedestrian goes from place A to place B, the shortest path may pass through 5 cameras, but according to the most probable path of the adjacent cameras, it may pass through 6 cameras or more cameras. However, a prominent function of the present application is to judge whether the behavior of the pedestrian is abnormal. If the predicted action trajectory of the pedestrian is to pass through 6 cameras, but it is actually found that the pedestrian passes through dozens of cameras and completely violates the predicted most probable path, this is obviously an intentional avoidance behavior or a completely abnormal behavior, and prevention is required at this time.

[0061] Therefore, the method of the present application further includes:

[0062] Obtain the distance information between adjacent cameras among the cameras passed by the estimated action trajectory of the pedestrian at the starting and ending positions; specifically, by obtaining the distance information between two cameras, the walking time between the two cameras can be obtained in combination with the speed of the pedestrian, and the behavior of the pedestrian can be judged whether it is normal walking through the walking time.

[0063] According to the distance information between the adjacent cameras and the traveling speed information of the pedestrian, obtain the arrival time information of each camera passed by the estimated action trajectory of the pedestrian at the starting and ending positions;

[0064] Based on the arrival time information of each camera passed by the estimated movement trajectory of the pedestrian at the starting and ending positions, and the actual implementation information of the pedestrian arriving at the corresponding camera, obtain the abnormal behavior information of the pedestrian.

[0065] According to the solution of pedestrian trajectory estimation based on crowd heat map of the present invention, by obtaining the spatial topology map of the camera layout, determine the starting and ending camera points and time of the pedestrian to be recognized; based on the spatial topology map of the camera layout and the starting and ending camera points and time of the pedestrian to be recognized, obtain the crowd heat map of each camera in the spatial topology map within the starting and ending camera points and time of the pedestrian to be recognized; based on the crowd heat map of each camera, obtain the maximum probability path of pedestrians passing between any camera and its adjacent camera; based on the maximum probability path of pedestrians passing between any camera and its adjacent camera, obtain the estimated movement trajectory of the pedestrian at the starting and ending positions. This application obtains the maximum probability path between each camera and its adjacent camera of the pedestrian between the starting and ending positions through the crowd heat map of the camera, and then obtains the estimated movement trajectory of the pedestrian, which can solve the problem of the failure of the pedestrian trajectory tracking at some points in the topology map caused by detection errors due to factors such as light, clothing, and wearing masks. For abnormal behavior personnel who do not travel according to the estimated movement trajectory, early warning can be achieved to prevent dangerous events from occurring.

[0066] It should be understood that although Figure 2 the steps in the flowchart of Figure 2 are shown in sequence according to the indication of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover,

[0067] In one embodiment, as Figure 3 shown, a pedestrian trajectory estimation device 300 based on crowd heat map is provided. The device 300 includes: an acquisition module, a heat map calculation module, and a path calculation module;

[0068] The acquisition module is used to obtain the spatial topology map of the camera layout and determine the starting and ending camera points and time of the pedestrian to be recognized;

[0069] A heat map calculation module, configured to obtain a crowd heat map of each camera in the spatial topology map according to the spatial topology map of the camera layout and the starting and ending camera points and time of the pedestrians to be recognized;

[0070] A path calculation module, configured to obtain the most probable path of pedestrians passing between any camera and its adjacent camera according to the crowd heat map of each camera; and obtain the estimated movement trajectory of the pedestrians at the starting and ending positions according to the most probable path of pedestrians passing between any camera and its adjacent camera.

[0071] Specifically, in another embodiment of the present application, the obtaining module is configured to obtain camera information and road information within a set area; obtain adjacent information of the cameras within the set area according to the camera information and road information within the set area; obtain a spatial topology map of the camera layout according to the adjacent information of the cameras within the set area; and obtain the starting and ending camera points of the pedestrians to be recognized and the position information of the camera points in the spatial topology map according to the spatial topology map of the camera layout.

[0072] Specifically, in another embodiment of the present application, the heat map calculation module is configured to obtain the pedestrian quantity information collected by each camera in the spatial topology map within a set time interval; obtain the crowd heat map of each camera in the spatial topology map according to the pedestrian quantity information collected by each camera in the spatial topology map within the set time interval; and obtain the cameras passed by when walking at the starting and ending camera points of the pedestrians to be recognized and the crowd heat map of each camera within the time to be recognized according to the crowd heat map of each camera in the spatial topology map and the starting and ending camera points and time of the pedestrians to be recognized.

[0073] Specifically, in another embodiment of the present application, the path calculation module is configured to obtain the crowd heat maps of any two adjacent cameras on the same road according to the crowd heat map of each camera and the road information; obtain the weight of pedestrians passing between two adjacent cameras on the same road according to the crowd heat maps of any two adjacent cameras on the same road; and obtain the most probable path of pedestrians passing between any camera and its adjacent camera according to the weight of pedestrians passing between two adjacent cameras on the same road.

[0074] Specifically, in another embodiment of the present application, the path calculation module is configured to compare the crowd heat maps of any two adjacent cameras on the same road; if the crowd heat maps of any two adjacent cameras on the same road are the same, the weight of pedestrians passing between the two adjacent cameras on the same road is 1; if the crowd heat maps of any two adjacent cameras on the same road are different, obtain the crowd heat map information of one of the cameras and all its adjacent cameras; according to the weights of pedestrians passing between the one camera and all its adjacent cameras, the sum of the weights of pedestrians passing between the camera with a larger heat map value and all its adjacent cameras is 1.

[0075] Specifically, in another embodiment of the present application, the path calculation module is configured to obtain the maximum probability path of pedestrians passing through the camera at the starting position of the pedestrian according to the maximum probability path of pedestrians passing between any camera and its adjacent cameras and the starting and ending camera positions of the pedestrian to be identified, and predict the next camera that the pedestrian will pass through; according to the maximum probability path of pedestrians passing between any camera and its adjacent cameras, sequentially predict each camera that the pedestrian will pass through; according to each predicted camera that the pedestrian will pass through, obtain the estimated movement trajectory of the pedestrian at the starting and ending positions.

[0076] It should be understood that, in order to streamline the present disclosure and assist in understanding one or more of the various inventive aspects, in the foregoing description of the exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be construed as reflecting the intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, the inventive aspects lie in less than all the features of the preceding single embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate embodiment of the present invention.

[0077] Those skilled in the art should understand that the modules, units, or components of the devices in the examples disclosed herein may be arranged in the devices as described in this embodiment, or alternatively may be located in one or more devices different from the devices in this example. The modules in the foregoing examples may be combined into one module or further divided into multiple sub-modules.

[0078] Those skilled in the art can understand that the modules in the devices in the embodiments can be adaptively changed and set in one or more devices different from the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and in addition, they can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all the features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all the processes or units of any method or device so disclosed. Unless otherwise explicitly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) can be replaced by an alternative feature that provides the same, equivalent, or similar purpose.

[0079] In addition, those skilled in the art can understand that although some of the embodiments described herein include certain features included in other embodiments rather than other features, the combination of the features of different embodiments means that it is within the scope of the present invention and forms different embodiments. For example, in the following claims, any one of the claimed embodiments can be used in any combination.

[0080] In addition, some of the embodiments herein are described as a combination of methods or method elements that can be implemented by a processor of a computer system or by other devices that perform the functions. Therefore, a processor having the necessary instructions for implementing the method or method element forms a device for implementing the method or method element. In addition, the elements described herein in the device embodiments are examples of such devices: the device is used to implement the functions performed by the elements for the purpose of implementing the invention.

[0081] As used herein, unless otherwise specified, the use of ordinal numbers "first", "second", "third", etc. to describe ordinary objects only indicates different instances of similar objects, and does not intend to imply that the objects so described must have a given order in terms of time, space, sorting, or in any other way.

[0082] Although the present invention has been described in terms of a limited number of embodiments, those skilled in the art, having the benefit of the foregoing description, will appreciate that other embodiments can be contemplated within the scope of the invention as thus described. Further, it should be noted that the language used in the present specification has been principally selected for readability and instructional purposes and not to limit or circumscribe the inventive subject matter. Accordingly, many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the appended claims. For the scope of the present invention, the disclosure herein is illustrative, and not restrictive, the scope of the invention being defined by the appended claims.

Claims

1. A pedestrian trajectory estimation method based on a crowd heat map. This method estimates the pedestrian trajectory by obtaining the heat map of the personnel distribution under the camera and calculating the weights of the paths between adjacent cameras. The method includes the following steps: Obtain the spatial topology map of the camera layout, and determine the starting and ending camera points and time of the pedestrians to be recognized; According to the spatial topology map of the camera layout and the starting and ending camera points and time of the pedestrians to be recognized, obtain the crowd heat map of each camera in the spatial topology map during the starting and ending camera points and time of the pedestrians to be recognized; According to the crowd heat map of each camera, obtain the most probable path passed by pedestrians between any camera and its adjacent camera, including obtaining the crowd heat maps of any two adjacent cameras on the same road according to the crowd heat map of each camera and the road information, obtaining the weights of the paths passed by pedestrians between two adjacent cameras on the same road according to the crowd heat maps of any two adjacent cameras on the same road, and obtaining the most probable path passed by pedestrians between any camera and its adjacent camera according to the weights of the paths passed by pedestrians between two adjacent cameras on the same road; According to the most probable path passed by pedestrians between any camera and its adjacent camera, obtain the estimated action trajectory of the pedestrians at the starting and ending positions.

2. The method according to claim 1, wherein, The step of obtaining the spatial topology map of the camera layout and determining the starting and ending camera points of the pedestrians to be recognized includes: Obtain the camera information and road information within the set area; According to the camera information and road information within the set area, obtain the adjacent information of the cameras within the set area; According to the adjacent information of the cameras within the set area, obtain the spatial topology map of the camera layout; According to the spatial topology map of the camera layout, obtain the starting and ending camera points of the pedestrians to be recognized, and the position information of the camera points in the spatial topology map.

3. The method according to claim 1, wherein The step of obtaining the crowd heat map of each camera in the spatial topology map during the starting and ending camera points and time of the pedestrians to be recognized according to the spatial topology map of the camera layout and the starting and ending camera points and time of the pedestrians to be recognized includes: Obtain the pedestrian quantity information collected by each camera in the spatial topology map within the set time interval; According to the pedestrian quantity information collected by each camera in the spatial topology map within the set time interval, obtain the crowd heat map of each camera in the spatial topology map; According to the crowd heat map of each camera in the spatial topology map and the starting and ending camera points and time of the pedestrians to be recognized, obtain the cameras passed by the pedestrians when walking at the starting and ending camera points to be recognized, and the crowd heat map of each camera during the time to be recognized.

4. The method according to claim 1, wherein The step of obtaining the weights of the paths passed by pedestrians between two adjacent cameras on the same road according to the crowd heat maps of any two adjacent cameras on the same road includes: Compare the crowd heat maps of any two adjacent cameras on the same road; If the crowd heat maps of any two adjacent cameras on the same road are the same, the weight of pedestrians passing between the two adjacent cameras on the same road is 1; If the crowd heat maps of any two adjacent cameras on the same road are different, obtain the crowd heat map information of one of the cameras and all its adjacent cameras; According to the weights of pedestrians passing between one of the cameras and all its adjacent cameras, the sum of the weights of pedestrians passing between the camera with the larger heat map weight and all its adjacent cameras is 1.

5. The method according to claim 1, wherein, The step of obtaining the estimated movement trajectory of a pedestrian at the starting and ending positions according to the maximum probability path of pedestrians passing between any camera and its adjacent cameras includes: According to the maximum probability path of pedestrians passing between any camera and its adjacent cameras, and the starting and ending camera points of the pedestrian to be recognized, obtain the maximum probability path of pedestrians passing through the camera at the starting position of the pedestrian, and predict the next camera that the pedestrian passes through; According to the maximum probability path of pedestrians passing between any camera and its adjacent cameras, sequentially predict each camera that the pedestrian passes through; According to each predicted camera that the pedestrian passes through, obtain the estimated movement trajectory of the pedestrian at the starting and ending positions.

6. The method according to claim 5, wherein, The steps of the method further include: Obtain the distance information between adjacent cameras among the cameras passed by the estimated movement trajectory of the pedestrian at the starting and ending positions; According to the distance information between adjacent cameras and the movement speed information of the pedestrian, obtain the arrival time information of each camera passed by the estimated movement trajectory of the pedestrian at the starting and ending positions; According to the arrival time information of each camera passed by the estimated movement trajectory of the pedestrian at the starting and ending positions, and the actual implementation information of the pedestrian arriving at the corresponding camera, obtain the abnormal behavior information of the pedestrian.

7. A pedestrian trajectory estimation device based on crowd heat maps. The device estimates the pedestrian trajectory by obtaining the heat map of the personnel distribution under the camera and calculating the weights of the paths of pedestrians passing between adjacent cameras. The device includes: An acquisition module, configured to acquire the spatial topology map of the camera layout, and determine the starting and ending camera points and time of the pedestrian to be recognized; A heat map calculation module, configured to obtain the crowd heat map of each camera in the spatial topology map within the starting and ending camera points and time of the pedestrian to be recognized according to the spatial topology map of the camera layout and the starting and ending camera points and time of the pedestrian to be recognized; A path calculation module, configured to obtain the maximum probability path of pedestrians passing between any camera and its adjacent camera according to the crowd heat map of each camera, including obtaining the crowd heat maps of any two adjacent cameras on the same road according to the crowd heat map of each camera and the road information, obtaining the weight of pedestrians passing between two adjacent cameras on the same road according to the crowd heat maps of any two adjacent cameras on the same road, obtaining the maximum probability path of pedestrians passing between any camera and its adjacent camera according to the weight of pedestrians passing between two adjacent cameras on the same road; and obtaining the estimated movement trajectory of pedestrians at the starting and ending positions according to the maximum probability path of pedestrians passing between any camera and its adjacent camera.

8. An electronic device, comprising: One or more processors; And A memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any one of the methods according to claims 1-6.

9. A computer-readable storage medium storing one or more programs, the one or more programs including instructions that, when executed by a computing device, cause the computing device to execute any one of the methods according to claims 1-6.

Images