A personnel entry and exit statistics method and system

Abstract

The present invention discloses a method and system for counting people entering and leaving, the method comprising receiving a video frame to be detected of a target area acquired by an image acquisition device; using a target detection model to acquire personnel information in the video frame to be detected, the personnel information including the position of the personnel and the entry and exit status of the personnel; and judging whether the position of the personnel is located at a preset counting position. The method does not rely on a tracking model, but uses a target detection model to detect the head of a person in real time, thereby realizing real-time classification of the entry and exit information of the personnel. An electronic bar for counting the number of people entering and an electronic bar for counting the number of people leaving are set, and the number of people entering and leaving is counted respectively by using the method of people breaking into the electronic bar. Real-time statistics can be realized on an embedded device, and scenes with a large number of people entering and leaving can be handled, and the counting speed will hardly decrease. Scenes with complex backgrounds can be handled, and it is not easy to misjudge or miss.

Description

Technical Field

[0001] The present invention relates to the technical field of personnel entry and exit data statistics, and in particular to a personnel entry and exit statistics method and system suitable for use with embedded devices and capable of realizing real-time large-volume statistics. Background Art

[0002] The problem of counting people in and out refers to counting the number of people entering and leaving a certain area within a certain period of time. The algorithm that solves this kind of problem is called the people in and out counting algorithm.

[0003] In the prior art, personnel entry and exit statistical algorithms are often developed based on tracking algorithms. Since tracking algorithms involve processes such as feature matching, the amount of data calculation is large, and its complexity is exponentially related to the number of tracked personnel. Therefore, it is usually used on servers equipped with sufficient computing resources.

[0004] In the post-PC era of rapid development of digital information technology and network technology, embedded systems have penetrated into various fields such as industry, agriculture, education, national defense, scientific research and daily life due to their many advantages such as small size, high reliability, strong functions, flexibility and convenience, and have played an extremely important role in promoting technological transformation, product upgrading, accelerating the automation process and improving productivity in various industries. However, due to the low computing power of embedded systems, it is impossible to provide sufficient computing resources to directly apply the personnel entry and exit statistics algorithm developed based on the tracking algorithm in the existing technology. In order to use embedded systems to realize personnel entry and exit statistics, the existing technology often uses simpler pedestrian detection and tracking algorithms, such as extracting pedestrian features through machine learning algorithms (HOG features, etc.) and performing rapid pedestrian detection through embedded systems.

[0005] However, although the personnel entry and exit statistics algorithm based on the traditional machine learning algorithm can be used on embedded devices, in order to achieve better results, the traditional machine learning algorithm requires a lot of parameter adjustment work, which greatly increases the threshold for use. At the same time, in the case of complex backgrounds, the traditional machine learning algorithm has more false positives and negatives, and the effect is poor. In addition, in the case of a large flow of people, even if a simpler pedestrian detection and tracking algorithm is used, the embedded system still runs very slowly when performing personnel entry and exit statistics, and cannot perform real-time statistics of a large flow of people entering and leaving. Summary of the invention

[0006] In view of the above problems, the present invention provides a method and system for counting people entering and leaving the space, which is used to overcome the above problems or at least partially solve the above problems. Instead of using a tracking algorithm to distinguish between people entering and leaving the space, the target detection model is used to directly provide information about people entering and leaving the space. After obtaining the information about people entering and leaving the space, the people are counted accordingly.

[0007] The present invention provides the following scheme:

[0008] A method for counting people in and out, comprising:

[0009] Receiving a video frame to be detected of a target area acquired by an image acquisition device;

[0010] Using a target detection model to obtain personnel information in the video frame to be detected, the personnel information includes the position of the personnel and the entry and exit status of the personnel;

[0011] It is determined whether the position of the person is located at a preset counting position, and if so, counting is triggered so as to add the number of people entering and leaving the person according to the state of the person.

[0012] Preferably: the target detection model includes a yolov4-tiny target detection model and a tensorrt reasoning framework.

[0013] Preferably: the detection object of the yolov4-tiny target detection model includes a person's head; the person position acquisition method includes:

[0014] Obtaining the detection frame position information corresponding to the person's head in the video frame to be detected;

[0015] The position of the person is determined according to the detection frame position information.

[0016] Preferably, the detection frame position information includes the horizontal and vertical coordinates of the vertex of the upper left corner of the detection frame and the horizontal and vertical coordinates of the vertex of the lower right corner of the detection frame.

[0017] Preferably: the camera of the image acquisition device faces the entrance and exit direction from the inside to the outside; the method for acquiring the entry and exit status of personnel includes:

[0018] Acquire detection frame category information corresponding to the person's head in the video frame to be detected, wherein the detection frame category information includes the front face and the back of the head;

[0019] The entry and exit status of the person is determined according to the detection frame category information.

[0020] Preferably, the determining the entry and exit status of the person according to the detection frame category information includes:

[0021] When the detection frame category information is a front face, determining that the person's entry and exit state is entering;

[0022] When the detection frame category information is the back of the head, it is determined that the person's entry and exit status is leaving.

[0023] Preferably, the counting position includes entering the electronic fence and leaving the electronic fence.

[0024] Preferably, the step of adding the number of in-and-out personnel statistics according to the personnel status includes:

[0025] If it is determined that the entry and exit status of the person is entered and the person's position is within the entry electronic column, the number of entered persons is increased by 1;

[0026] If it is determined that the entry / exit status of the person is leaving and the position of the person is located in the leaving electronic column, the number of the leaving persons is increased by 1.

[0027] A personnel entry and exit statistics system, the system comprising:

[0028] A video frame acquisition module, used to receive the video frame to be detected of the target area acquired by the image acquisition device;

[0029] A personnel information detection module, used to obtain personnel information in the video frame to be detected by using a target detection model, wherein the personnel information includes the position of the personnel and the entry and exit status of the personnel;

[0030] The personnel counting module is used to determine whether the position of the personnel is located at a preset counting position, and if so, trigger counting so as to add the number of personnel entering and leaving the personnel accordingly according to the personnel status.

[0031] Preferably: the personnel information detection module includes a Yolov4-tiny detection submodule and a personnel information extraction submodule.

[0032] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0033] The embodiment of the present application provides a method and system for counting people entering and leaving. The method does not rely on a tracking model, but uses a target detection model to detect people's heads in real time, thereby realizing real-time classification of people's entry and exit information. An electronic column for counting people entering and an electronic column for counting people leaving are set up, and people breaking into the electronic column are used to count people entering and leaving respectively. Real-time statistics can be realized on embedded devices, and can cope with scenes with a large number of people entering and leaving, and the counting speed will hardly decrease. It can cope with scenes with complex backgrounds and is not prone to misjudgment or missed judgment.

[0034] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 This is a flow chart of a personnel entry and exit statistics method provided by an embodiment of the present invention;

[0037] Figure 2 It is a schematic diagram of a personnel entry and exit statistics system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. Based on the embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0039] See also Figure 1 , is a personnel entry and exit statistics method provided by an embodiment of the present invention, such as Figure 1 As shown, the method may include:

[0040] S101: receiving a video frame of a target area to be detected acquired by an image acquisition device; the image acquisition device may be a common surveillance camera, and its image acquisition range may be set as required to collect images within the target range. For example, a corridor connecting a doorway in a building and a room may be used as the target detection range.

[0041] S102: using a target detection model to obtain personnel information in the to-be-detected video frame, where the personnel information includes the personnel position and the personnel entry and exit status;

[0042] S103: Determine whether the position of the person is located at a preset counting position, and if so, trigger counting so as to add the number of people entering and leaving the person according to the state of the person.

[0043] The personnel entry and exit statistical algorithm needs to count the number of people entering and leaving. The most important thing is to distinguish between the people entering and leaving, and count them correctly. Since the calculation bottleneck of the existing personnel entry and exit statistical algorithm lies in the tracking algorithm, the personnel entry and exit statistical method provided in the embodiment of the present application does not use the tracking algorithm to distinguish between the people entering and leaving, but directly provides the entry and exit information of the people through the target detection model. After obtaining the information of the people entering and leaving, the corresponding number of the people is counted.

[0044] In practical applications, the target detection model can take various forms. For example, in one implementation, the embodiment of the present application can provide that the target detection model includes a yolov4-tiny target detection model and a tensorrt reasoning framework.

[0045] The detection object of the yolov4-tiny target detection model includes a person's head; the method for obtaining the person's position includes:

[0046] Obtaining the detection frame position information corresponding to the person's head in the video frame to be detected;

[0047] The position of the person is determined according to the detection frame position information.

[0048] After detecting a person's head, the yolov4-tiny target detection model will generate a rectangular detection frame around the person's head, and use the location information of the detection frame as the person's actual location information, so that the person's location can be judged.

[0049] Specifically, the detection frame position information includes the horizontal and vertical coordinates of the top left corner vertex of the detection frame and the horizontal and vertical coordinates of the bottom right corner vertex of the detection frame. The detailed method and principle for determining the detection frame position information will be described in detail later.

[0050] By acquiring the position information of the detection frame generated by the head of the person, the current position of the person can be determined, which is used to subsequently determine whether it meets the requirements of adding and subtracting the number of people entering and leaving the person. In order to judge the current entry and exit status of the person, the embodiment of the present application can also provide that the camera of the image acquisition device faces the direction of the entrance and exit from the inside to the outside; the method for acquiring the entry and exit status of the person includes:

[0051] Acquire detection frame category information corresponding to the person's head in the video frame to be detected, wherein the detection frame category information includes the front face and the back of the head;

[0052] The entry and exit status of the person is determined according to the detection frame category information.

[0053] By analyzing the head image of the person in the detection frame, it can be determined whether the person's front face is facing the camera or the back of the head is facing the camera. Combined with the installation direction of the camera, it can be determined whether the person is currently in the entry state or the exit state. Specifically, the determination of the entry and exit state of the person according to the detection frame category information includes:

[0054] When the detection frame category information is a front face, determining that the person's entry and exit state is entering;

[0055] When the detection frame category information is the back of the head, it is determined that the person's entry and exit status is leaving.

[0056] In order to prevent over-counting, the embodiment of the present application may further provide that the counting position includes entering the electronic fence and leaving the electronic fence.

[0057] Specifically, the step of adding the number of in-and-out personnel statistics according to the personnel status includes:

[0058] If it is determined that the entry and exit status of the person is entered and the person's position is within the entry electronic column, the number of entered persons is increased by 1;

[0059] If it is determined that the entry / exit status of the person is leaving and the position of the person is located in the leaving electronic column, the number of the leaving persons is increased by 1.

[0060] In summary, the personnel entry and exit statistics method provided in the embodiment of the present application does not rely on a tracking model, but uses a target detection model to detect personnel heads in real time, thereby realizing real-time classification of personnel entry and exit information. An electronic column for counting incoming personnel and an electronic column for counting outgoing personnel are set up, and the way in which personnel break into the electronic column is used to count incoming personnel and outgoing personnel respectively. Real-time statistics can be realized on an embedded device (jetson nano), and it can cope with scenes with a large number of people entering and exiting, and the statistical speed will hardly decrease. It can cope with scenes with complex backgrounds, and is not prone to misjudgment or missed judgment.

[0061] See also Figure 2 , which corresponds to a personnel entry and exit statistics method provided in an embodiment of the present application, such as Figure 2 As shown, the embodiment of the present application also provides a personnel entry and exit statistics system, which may specifically include:

[0062] The video frame acquisition module 201 is used to receive the video frame to be detected of the target area acquired by the image acquisition device;

[0063] A personnel information detection module 202 is used to obtain personnel information in the video frame to be detected by using a target detection model, wherein the personnel information includes the position of the personnel and the entry and exit status of the personnel;

[0064] The personnel counting module 203 is used to determine whether the position of the personnel is located at a preset counting position, and if so, trigger counting so as to add the number of in-and-out personnel statistics accordingly according to the personnel status.

[0065] Specifically, the personnel information detection module includes a Yolov4-tiny detection submodule and a personnel information extraction submodule.

[0066] The personnel entry and exit counting method and system provided in the embodiments of the present application are introduced in detail below.

[0067] The method provided in the embodiment of the present application receives video frames from the outside, and the video frames are transmitted to the personnel information detection module. In the personnel information detection module, the video frames are processed by a specific target detection algorithm to obtain the personnel information in the video frames, which includes the personnel position, personnel entry or exit information, etc. The processed personnel information is sent to the personnel counting module. In this module, in order to avoid multiple counting of personnel, counting can only be triggered when the personnel position meets specific requirements. According to the personnel entry and exit information, the entry and exit statistics are added accordingly, and finally the statistical results are output in real time.

[0068] Each module will be described in detail below.

[0069] (1) Personnel information detection module

[0070] In the personnel information detection module, the video frame to be detected is input externally, and the required personnel information is obtained after processing. Specifically, this module contains the Yolov4-tiny detection submodule and the personnel information extraction submodule. In the Yolov4-tiny detection submodule, the video frame is passed through a specially trained Yolov4-tiny detection model to obtain the detection result; in the personnel information extraction submodule, we process the detection result to obtain the required personnel information.

[0071] The two sub-modules will be explained below.

[0072] In the Yolov4-tiny detection submodule, in order to achieve the fastest speed and better detection effect on an embedded device (here Jetson Nano is selected), the method provided in this application selects the Yolov4-tiny target detection model and uses the TensorRT reasoning framework for acceleration. In order to use the Yolov4-tiny target detection model to give the location of personnel and the information of personnel entering and exiting, the model needs to be retrained.

[0073] The training process is as follows:

[0074] a. Training data: about 700 images taken by cameras at entrances, exits or corridors. The images must have clear head features and be either walking out or entering. Images should not contain side faces or images where it is difficult to determine the direction of entry or exit.

[0075] b. Data labeling: The labeling categories are the front and back of the head. When labeling, be careful to avoid labeling heads that are too far away or have too small pixel sizes. For example, in a 1280*720 image, the minimum labeling size should not be less than 50*50.

[0076] c. Data training: Use darknet training data, set the number of rounds to 10,000, decrease the learning rate by 0.1 in the 8,000th and 9,000th rounds respectively, and set the image input to 640*640 to maximize the training effect of small objects.

[0077] After the above training process, the yolov4-tiny detection model is obtained, and the detection accuracy can reach more than 99%. The detection model can output the detection frame information (x1, y1, x2, y2) and category information id of each person in the picture frame, where (x1, y1) is the horizontal and vertical coordinates of the vertex in the upper left corner of the detection frame, (x2, y2) is the horizontal and vertical coordinates of the vertex in the lower right corner of the detection frame, and id is the category (front face or back of the head).

[0078] In summary, in the Yolov4-tiny detection submodule, the input and output are shown in Table 1.

[0079] Table 1 Input and output table of Yolov4-tiny detection submodule

[0080]

[0081]

[0082] In the personnel information extraction submodule, the required personnel information can be extracted from the detection results output by the Yolov4-tiny detection submodule. The personnel information here includes personnel location information and personnel entry and exit information.

[0083] The specific extraction steps are:

[0084] a. Extract the person's position information from the detection frame position information. Since the detection frame is the head part and is relatively small, the midpoint of the detection frame is used to represent the head position, and then the person's position. Therefore, the conversion formula from the detection frame position information (x1, y1, x2, y2) to the person's position information (centerx, centery) is:

[0085] centerx=(x1+x2) / 2

[0086] centery=(y1+y2) / 2

[0087] b. Extract person category information from the detection frame category information. The detection frame category information includes the front face or the back of the head. In general, the camera faces the entrance and exit from the inside to the outside. The head of the person entering is the front face, and the head of the person leaving is the back of the head.

[0088] Therefore, this application may stipulate that when the detection frame category information is the front face, the person is entering; and when the detection frame category information is the back of the head, the person is leaving.

[0089] In summary, in the personnel information extraction submodule, the input and output are shown in Table 2.

[0090] Table 2 Input and output table of personnel information extraction submodule

[0091]

[0092] In summary, in the personnel information detection module, the picture frame is processed by the Yolov4-tiny detection submodule and the personnel information extraction submodule to obtain the required personnel information, including personnel location information and personnel entry and exit information.

[0093] (2) People counting module

[0094] In the personnel counting and statistics module, personnel information is input externally, and the required statistical results are obtained after a series of judgments and processing.

[0095] The specific steps are as follows:

[0096] a. Before making judgments and processing, first make the electronic bar In and the electronic bar Out, which are used to count the people entering and leaving respectively. When the person's entry and exit information is the position of the person entering (the detection category is front) and is located in the electronic bar In, the entry person will be counted plus 1; when the person's entry and exit information is the position of the person leaving (the detection category is back) and is located in the electronic bar Out, the exit person will be counted plus 1. Here, you need to pay attention to setting the width and height of the electronic bar to prevent excessive counting.

[0097] b. According to the personnel entry and exit information in the externally input personnel information, determine whether the current person is entering or leaving. If entering, determine whether it is in the electronic column In according to the position information in the personnel information. If so, the number of entering personnel statistics increases by 1, if not, the statistics remain unchanged. If the current person is judged to be leaving, determine whether it is in the electronic column Out according to the position information in the personnel information. If so, the number of leaving personnel statistics increases by 1, if not, the statistics remain unchanged. The input and output table of the personnel counting statistics module is shown in Table 3.

[0098] Table 3 Input and output table of personnel counting statistics module

[0099]

[0100] In summary, in the personnel counting statistics module, the input and output are shown in Table 3. After the personnel information passes through this module, the statistical results can be generated.

[0101] At this point, after receiving the picture frame, the real-time high-flow personnel entry and exit statistics algorithm can output the personnel entry and exit statistics results in real time after being processed by the personnel information detection module and the personnel counting statistics module.

[0102] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0103] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0104] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0105] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method for counting people in and out, characterized in that: The method comprises: Receiving a video frame to be detected of a target area acquired by an image acquisition device; Using a target detection model to obtain personnel information in the video frame to be detected, the personnel information includes the position of the personnel and the entry and exit status of the personnel; Determine whether the position of the person is at a preset counting position, and if so, trigger counting so as to add the number of people entering and leaving the person accordingly according to the entry and exit status of the person; The target detection model includes a yolov4-tiny target detection model and a tensorrt reasoning framework; the detection object of the yolov4-tiny target detection model includes a person's head; the person position acquisition method includes: Obtaining the detection frame position information corresponding to the person's head in the video frame to be detected; Determine the position of the person according to the detection frame position information; The camera of the image acquisition device faces the entrance and exit from the inside to the outside; the method for acquiring the entry and exit status of personnel includes: Acquire detection frame category information corresponding to the person's head in the video frame to be detected, wherein the detection frame category information includes the front face and the back of the head; Determining the entry and exit status of the person according to the detection frame category information; The counting position includes entering and leaving the electronic fence; and the adding of the number of people entering and leaving the electronic fence according to the entry and exit status of the people includes: If it is determined that the entry and exit status of the person is entered and the person's position is within the entry electronic column, the number of entered persons is increased by 1; If it is determined that the entry / exit status of the person is leaving and the position of the person is located in the leaving electronic column, the number of the leaving persons is increased by 1.

2. The personnel entry and exit counting method according to claim 1 is characterized in that: The detection frame position information includes the horizontal and vertical coordinates of the vertex of the upper left corner of the detection frame and the horizontal and vertical coordinates of the vertex of the lower right corner of the detection frame.

3. The personnel entry and exit counting method according to claim 1 is characterized in that: The determining the entry and exit status of the person according to the detection frame category information includes: When the detection frame category information is a front face, determining that the person's entry and exit state is entering; When the detection frame category information is the back of the head, it is determined that the person's entry and exit status is leaving.

4. A personnel entry and exit statistics system, characterized in that: Used to execute the personnel entry and exit counting method according to any one of claims 1 to 3, the system comprises: A video frame acquisition module, used to receive the video frame to be detected of the target area acquired by the image acquisition device; A personnel information detection module, used to obtain personnel information in the video frame to be detected by using a target detection model, wherein the personnel information includes the position of the personnel and the entry and exit status of the personnel; The personnel counting module is used to determine whether the position of the personnel is located at a preset counting position, and if so, trigger counting so as to add the number of personnel entering and leaving the personnel accordingly according to the personnel entering and leaving status.

5. The personnel entry and exit statistics system according to claim 4, characterized in that: The personnel information detection module includes a Yolov4-tiny detection submodule and a personnel information extraction submodule.

Images