A method for counting the personnel flow in a scene, a terminal device, and a storage medium

Abstract

The present invention relates to a method for counting the flow of personnel in a scene, a terminal device, and a storage medium. The method includes: performing human head and shoulder detection on a top-down perspective human body image to be detected through a human head and shoulder detection model to obtain detection frames of human heads and shoulders; calculating the IOU distances between the detection frames of all human heads and shoulders in the current frame and the detection frames of all human heads and shoulders in the previous frame, and matching the detection frames of human heads and shoulders in the current frame and the previous frame based on the calculated IOU distances to obtain all human trajectories; updating the position states of the current frames of each human trajectory relative to the scene area; and updating the entry and exit states of the current frames of each human trajectory relative to the scene area based on the updated position states. The present invention improves the accuracy of personnel flow statistics and reduces the influence of jitter of the neural network detection frames.

Description

Technical Field

[0001] The present invention relates to the field of personnel flow, and particularly to a method for counting the flow of people in a scene, a terminal device, and a storage medium. Background Art

[0002] The method of personnel flow statistics refers to the statistics of the entry and exit of people in a region and the entry and exit directions. The difficulty lies in accurately detecting human targets. Currently, there are many solutions for personnel flow statistics. While solving some problems, the following problems also exist:

[0003] (1) Detection method based on human body sensors: By attaching sensors to the human body to track the dynamic movement of the human body in real time, when the sensor enters or exits the detection area, the entry and exit of the human body are detected and counted. Such a method cannot determine whether the sensor signal source is a human body, and the scene is limited. It requires the human body to wear a specific detection device, causing inconvenience.

[0004] (2) Method based on target visual features: By using an image acquisition device to obtain real-time images of the detection area, and detecting the shape features of the human body in the image, such as the circular-like features of the human head shape, or establishing a human head color model in the HIS color space and combining the hair gray range to detect the human body. However, it is easy to misdetect when there are objects with similar features in the background. There is also a method of extracting high-dimensional features and then classifying, such as extracting the HOG features of the image and then using SVM classification for human head detection. This method has higher accuracy than other detection methods based on visual features. However, since the human body is a non-rigid body and the features are variable, this method will still have misdetection in some scenes where the features are not obvious.

[0005] (3) Pedestrian detection method based on deep learning: Using the current mainstream object detection network, extracting features of the whole human body, and then performing classification detection. This method has higher generalization ability and higher scene adaptability than other methods. However, in places with a large flow of people, when the human bodies overlap and block each other in the picture, it is easy to miss detection. Summary of the Invention

[0006] In order to solve the above problems, the present invention proposes a method for counting the flow of people in a scene, a terminal device, and a storage medium.

[0007] The specific solutions are as follows:

[0008] A method for counting the flow of people in a scene includes the following steps:

[0009] S1: Performing human head and shoulder detection on the top-down perspective human body image to be detected through a human head and shoulder detection model, and obtaining the detection frame of the human head and shoulder;

[0010] S2: Calculate the IOU distances between the detection bounding boxes of all human head and shoulders in the current frame and those in the previous frame, and match the detection bounding boxes of human head and shoulders in the current frame and the previous frame based on the calculated IOU distances to obtain all human trajectories;

[0011] S3: Update the position status of each human trajectory in the current frame relative to the scene area;

[0012] S4: Update the entry and exit status of each human trajectory in the current frame relative to the scene area based on the updated position status.

[0013] Further, step S2 also includes: adding the detection bounding boxes that fail to be matched successfully in the current frame to the hidden trajectory queue. Until a certain hidden trajectory in the hidden trajectory queue persists for a period of time, delete this hidden trajectory from the hidden trajectory queue and change it to a human trajectory.

[0014] Further, step S2 also includes: if there are human trajectories that fail to be matched successfully, set the number of unmatched times of it to increase by 1. Until the number of unmatched times is greater than the preset threshold, delete this human trajectory.

[0015] Further, in step S3, update the position status of each human trajectory in the current frame relative to the scene area: extract multiple points as detection anchor points within the detection bounding box of human head and shoulders in the current frame, and judge the position status of the current frame of the human trajectory relative to the scene area according to the number of detection anchor points located within the scene area among all detection anchor points.

[0016] Further, the method of extracting multiple points as detection anchor points within the detection bounding box of human head and shoulders in the current frame is: shrink the detection bounding box of human head and shoulders to obtain a candidate box, and extract multiple points as detection anchor points within the candidate box.

[0017] Further, the method of extracting multiple points as detection anchor points within the candidate box is: use the four corner points of the candidate box as detection anchor points.

[0018] Further, the method of judging the position status of the current frame of the human trajectory relative to the scene area according to the number of detection anchor points located within the scene area among all detection anchor points is: if all detection anchor points are located outside the scene area, it is determined that the current frame of this trajectory is outside the scene area; if some of the detection anchor points are located inside the scene area and some are located outside the scene area, it is determined that the current frame of this trajectory is in the state of entering or exiting the scene area; if all detection anchor points are located inside the scene area, it is determined that the current frame of this trajectory is located inside the scene area.

[0019] Further, the method for updating the entry and exit status in step S4 is:

[0020] When the trajectory state changes from being outside the scene area to the state of entering and exiting the scene area, and remains in the state of entering and exiting the scene area for n consecutive frames, it is determined that this trajectory enters the scene;

[0021] When the trajectory state changes from being outside the scene area to being inside the scene area, it is determined that this trajectory enters the scene;

[0022] When the trajectory state changes from the state of entering and exiting the scene area to being outside the scene area, it is determined that this trajectory exits the scene;

[0023] When the trajectory state changes from the state of entering and exiting the scene area to being inside the scene area, it is determined that this trajectory enters the scene;

[0024] When the trajectory state changes from being inside the scene area to the state of entering and exiting the scene area, and remains in the state of entering and exiting the scene area for n consecutive frames, it is determined that this trajectory exits the scene;

[0025] When the trajectory state changes from being inside the scene area to being outside the scene area, it is determined that this trajectory exits the scene.

[0026] A terminal device for counting the flow of people in a scene, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method described above in the embodiments of the present invention are implemented.

[0027] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described above in the embodiments of the present invention are implemented.

[0028] The present invention adopts the above technical solutions and has the following beneficial effects:

[0029] (1) Detecting the human body using a head and shoulders model from a top-down perspective circumvents the situation of human body overlap and occlusion to a certain extent, improves the accuracy of the model. At the same time, the visible area from the top-down perspective is parallel to the actual detection area, avoiding some problems that occur when detecting three-dimensional objects in a two-dimensional image.

[0030] (2) Using the relationship between the detection anchor points within the head and shoulders detection box and the scene area for detection to judge the position of the human body has the advantages of higher accuracy and sensitivity compared to using the relationship between the head and shoulders box and the detection area box, and reduces the influence of the jitter of the neural network detection box to a certain extent.

[0031] (3) Using a trajectory sequence to record the real-time state of the head and shoulders detection box can make the movement of the head and shoulders detection box smoother, reduce the influence of misdetection in individual frames on the result, and has stronger robustness. Description of the Drawings

[0032] Figure 1 The following is a flowchart of the first embodiment of the present invention. Specific implementation manners

[0033] To further illustrate each embodiment, the present invention provides accompanying drawings. These accompanying drawings are part of the disclosure of the present invention, mainly used to illustrate the embodiments, and can be used in conjunction with the relevant descriptions in the specification to explain the operating principle of the embodiments. With reference to these contents, those of ordinary skill in the art should be able to understand other possible implementation manners and the advantages of the present invention.

[0034] The present invention will be further described below in conjunction with the accompanying drawings and specific implementation manners.

[0035] Embodiment 1:

[0036] The embodiment of the present invention provides a method for counting the flow of people in a scene, as Figure 1 shown, the method includes the following steps:

[0037] S1: Perform human head and shoulder detection on the top-down perspective human image to be detected through a human head and shoulder detection model, and obtain the detection frame of the human head and shoulder.

[0038] The human image taken from the top-down perspective can be taken by a fish-eye camera. For example, in this embodiment, it is taken in the area where the flow of people to be detected is located, and the fish-eye camera can be installed at a height of 3 to 6 meters from the ground. The top-down perspective human image to be detected is the frame image in the video taken by the camera.

[0039] The human head and shoulder detection model needs to be trained in advance. In this embodiment, the SCRFD network is selected, and a key point (top of the head, shoulder key point) branch is added to improve the longitude of the network.

[0040] S2: Calculate the IOU distance between the detection frames of all human heads and shoulders in the current frame and the detection frames of all human heads and shoulders in the previous frame, and match the detection frames of the human heads and shoulders in the current frame and the previous frame based on the calculated IOU distance to obtain all human trajectories.

[0041] Furthermore, it also includes adding the detection frames that are not successfully matched in the current frame to the hidden trajectory queue. Until a certain hidden trajectory in the hidden trajectory queue lasts for a period of time, this hidden trajectory is deleted from the hidden trajectory queue and changed to a human trajectory. This situation is applicable to the case of newly added human targets, avoiding misjudgment in one frame or several frames and improving accuracy. The size of the period of time can be set by those skilled in the art themselves and will not be limited here.

[0042] Further, if a certain trajectory is updated more than N times in subsequent video frames, mark this trajectory as a real trajectory; if no update is obtained, it is a deletable trajectory. If there are unmatched existing human trajectories, set the number of unmatched times to increase by 1 until the number of unmatched times is greater than a preset threshold, and then delete this human trajectory. Those skilled in the art can set the values of N and the threshold by themselves. In this embodiment, they are set to 5.

[0043] S3: Update the position state of the current frame of each human trajectory relative to the scene area.

[0044] The position state update method adopted in this embodiment is as follows: Extract multiple points as detection anchor points within the detection box of the human head and shoulders in the current frame, and judge the position state of the current frame of the human trajectory relative to the scene area according to the number of detection anchor points located within the scene area among all detection anchor points.

[0045] In the selection of detection anchor points, in this embodiment, in order to ensure that the detection anchor points are located at the real human head and shoulders, the detection box of the human head and shoulders is shrunk (such as shrinking by one-sixth) to obtain a candidate box, and multiple points are extracted within the candidate box as detection anchor points. For the convenience of selection, in this embodiment, the four corner points of the candidate box are used as detection anchor points.

[0046] The method for judging the position state based on detection anchor points in this embodiment is as follows: The method for judging the position state of the current frame of the human trajectory relative to the scene area according to the number of detection anchor points located within the scene area among all detection anchor points is as follows: If all detection anchor points are located outside the scene area, it is determined that the current frame of this trajectory is outside the scene area; if some of the detection anchor points are located inside the scene area and some are located outside the scene area, it is determined that the current frame of this trajectory is in the state of entering and exiting the scene area; if all detection anchor points are located inside the scene area, it is determined that the current frame of this trajectory is located inside the scene area.

[0047] S4: Based on the updated position state, update the entry and exit state of the current frame of each human trajectory relative to the scene area.

[0048] The entry and exit state update method in this embodiment is as follows:

[0049] When the trajectory state changes from being outside the scene area to being in the state of entering and exiting the scene area and remains in the state of entering and exiting the scene area for n frames, it is determined that this trajectory enters the scene;

[0050] When the trajectory state changes from being outside the scene area to being inside the scene area, it is determined that this trajectory enters the scene;

[0051] When the trajectory state changes from being in the state of entering and exiting the scene area to being outside the scene area, it is determined that this trajectory exits the scene;

[0052] When the trajectory state changes from the state of entering or exiting the scene area to the state of being inside the scene area, it is determined that this trajectory enters the scene;

[0053] When the trajectory state changes from the state of being inside the scene area to the state of entering or exiting the scene area, and it remains in the state of entering or exiting the scene area for n frames, it is determined that this trajectory exits the scene;

[0054] When the trajectory state changes from the state of being inside the scene area to the state of being outside the scene area, it is determined that this trajectory exits the scene.

[0055] Those skilled in the art can set the value of n by themselves and there is no limitation here.

[0056] The embodiment of the present invention uses a trajectory sequence to record the real-time state of the head and shoulder detection frame, which can make the movement of the head and shoulder detection frame smoother, reduce the influence of misdetection of individual frames on the result, and has stronger robustness.

[0057] Embodiment 2:

[0058] The present invention also provides a terminal device for statistical analysis of personnel flow in a scene, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps in the above method embodiment of Embodiment 1 of the present invention are implemented.

[0059] Furthermore, as an executable solution, the terminal device for statistical analysis of personnel flow in a scene can be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The terminal device for statistical analysis of personnel flow in a scene may include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the above composition structure of the terminal device for statistical analysis of personnel flow in a scene is only an example of the terminal device for statistical analysis of personnel flow in a scene, and does not constitute a limitation on the terminal device for statistical analysis of personnel flow in a scene. It may include more or fewer components than the above, or combine some components, or different components. For example, the terminal device for statistical analysis of personnel flow in a scene may also include input and output devices, network access devices, buses, etc., and the embodiment of the present invention does not make any limitation on this.

[0060] Further, as an executable solution, the so-called processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The processor is the control center of the scene personnel flow statistics terminal device, and connects various parts of the entire scene personnel flow statistics terminal device through various interfaces and lines.

[0061] The memory can be used to store the computer program and / or module. By running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory, the processor realizes various functions of the scene personnel flow statistics terminal device. The memory mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as a hard disk, memory, plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, at least one magnetic disk storage device, flash device, or other volatile solid-state storage devices.

[0062] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the above method in the embodiments of the present invention are realized.

[0063] If the modules / units integrated in the terminal device for scenario personnel flow statistics are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-described embodiment methods of the present invention, it can also be completed by a computer program instructing relevant hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-described various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), and software distribution medium, etc.

[0064] Although the present invention has been specifically shown and described in conjunction with the preferred embodiments, those skilled in the art should understand that various changes can be made to the present invention in terms of form and details without departing from the spirit and scope of the present invention defined by the appended claims, and all such changes are within the protection scope of the present invention.

Claims

1. A method for counting the personnel flow in a scene, characterized in that, Including the following steps: S1: Perform human head and shoulder detection on the top-down perspective human image to be detected through a human head and shoulder detection model, and obtain the detection box of the human head and shoulder; S2: Calculate the IOU distance between the detection boxes of all human heads and shoulders in the current frame and the detection boxes of all human heads and shoulders in the previous frame, and match the detection boxes of the human heads and shoulders in the current frame and the previous frame based on the calculated IOU distance to obtain all human trajectories; S3: Update the position state of the current frame of each human trajectory relative to the scene area; the update process includes: extracting multiple points as detection anchor points within the detection box of the human head and shoulder in the current frame, and judging the position state of the current frame of the human trajectory relative to the scene area according to the number of detection anchor points located within the scene area among all detection anchor points. Specifically: if all detection anchor points are located outside the scene area, it is determined that the current frame of this trajectory is outside the scene area; if some of the detection anchor points are located inside the scene area and some are located outside the scene area, it is determined that the current frame of this trajectory is in the state of entering and exiting the scene area; if all detection anchor points are located inside the scene area, it is determined that the current frame of this trajectory is located inside the scene area; S4: Based on the updated position state, update the entry and exit state of each human trajectory's current frame relative to the scene area; the method for updating the entry and exit state is: When the trajectory state changes from being outside the scene area to being in the state of entering and exiting the scene area and remains in the state of entering and exiting the scene area for n frames, it is determined that this trajectory enters the scene; When the trajectory state changes from being outside the scene area to being inside the scene area, it is determined that this trajectory enters the scene; When the trajectory state changes from being in the state of entering and exiting the scene area to being outside the scene area, it is determined that this trajectory exits the scene; When the trajectory state changes from being in the state of entering and exiting the scene area to being inside the scene area, it is determined that this trajectory enters the scene; When the trajectory state changes from being inside the scene area to being in the state of entering and exiting the scene area and remains in the state of entering and exiting the scene area for n frames, it is determined that this trajectory exits the scene; When the trajectory state changes from being inside the scene area to being outside the scene area, it is determined that this trajectory exits the scene.

2. The method for counting the flow of personnel in a scenario according to claim 1, wherein: Step S2 also includes: adding the detection boxes that are not successfully matched in the current frame to the hidden trajectory queue. Until a certain hidden trajectory in the hidden trajectory queue lasts for a period of time, delete this hidden trajectory from the hidden trajectory queue and change it to a human trajectory.

3. The method for statistically analyzing the personnel flow in a scenario according to claim 1, wherein: Step S2 also includes: if there are unmatched existing human trajectories, set the number of unmatched times to increase by 1. Until the number of unmatched times is greater than the preset threshold, delete this human trajectory.

4. The method for counting the personnel flow in a scene according to claim 1, wherein: The method for extracting multiple points as detection anchor points within the detection box of the human head and shoulder in the current frame is: shrink the detection box of the human head and shoulder to obtain a candidate box, and extract multiple points as detection anchor points within the candidate box.

5. The method for statistically analyzing the personnel flow in a scene according to claim 4, wherein: The method for extracting multiple points as detection anchor points within the candidate box is: use the four corner points of the candidate box as detection anchor points.

6. A terminal device for counting the personnel flow in a scene, characterized in that: It includes a processor, a memory, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 5.

Images