Personnel entry and exit statistical method, device, equipment and medium

Abstract

The invention discloses a personnel access statistical method, device and equipment and a medium, and relates to the technical field of image processing, and the method is realized based on a lightweight algorithm, can be efficiently operated on edge computing equipment, and does not need to depend on a high-performance server, so that the complexity and deployment cost of a system are reduced. The method has low requirements on hardware resources and is suitable for various edge devices (such as intelligent cameras and embedded systems), so that the system is more convenient to install and expand in an actual factory and adapts to scenes of different scales. Meanwhile, the method can realize millisecond-level response speed, and ensures real-time monitoring of entering and exiting of personnel at a factory gate. Through real-time processing and rapid feedback of data, managers can master the trends of personnel inside and outside the factory in real time, thereby facilitating improvement of safety management and handling of emergencies.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, equipment and medium for counting entry and exit of personnel based on deep learning. Background Art

[0002] Personnel entry and exit management has always been an important part of work in all walks of life. For some highly controlled places, such as government agencies, corporate office buildings, military bases, etc., timely grasp of personnel entry and exit is crucial to maintaining order and ensuring safety.

[0003] Traditional personnel entry and exit management mainly relies on manual records, which have problems such as incomplete records. With the rise of "face recognition" and "card swiping", many places have replaced the manual recording method with gates instead of "doors", but many times the entry and exit of the venue is open, which is not convenient for the use of gates. In addition, infrared equipment is currently used to count the number of people entering and leaving the venue, but this method has obvious disadvantages. It is easy to count errors when multiple people walk side by side, and it is only suitable for single-person entry and exit environments. At present, monitoring can be seen everywhere in many scenarios. Using monitoring equipment to count the number of people entering and leaving is an economical and efficient way. It has the advantages of high real-time, high accuracy, and multi-person counting, and has broad application prospects.

[0004] Patent document "Automatic Passenger Counter CN1438609A" adopts the method of arranging infrared probes on both sides of the bus door, and uses the logical order of blocking the infrared probes in sequence during the process of getting on and off to judge getting on and off, and judges the number of people according to the width and thickness of the human body. However, this device has obvious shortcomings: inaccurate counting, easy interference in up and down judgment, difficult to adapt to the flow of people counting more than 3 people and a small scope of application. When 2 people are parallel and close to each other, it is easy to count 2 people as 1 person. When people get on and off at the same time, especially when there are many people, the infrared probe detection signals are easy to interfere with each other and cannot correctly distinguish between getting on and off. The arrangement of the infrared probes on both sides objectively determines that only 2 people can be judged. When there are more than 2 people, only two people will be counted, so it is not suitable for counting large-volume people entering and leaving.

[0005] Patent document "A monitoring system for detecting the number of people entering and exiting by infrared array CN203825691U" detects the number of people entering and exiting by an infrared array. The system includes a pyroelectric infrared sensor, a signal processing circuit, a power supply circuit, a serial interface circuit, a single-chip microcomputer and an overcrowding monitor. This monitoring system uses a pyroelectric infrared sensor to obtain infrared rays emitted by a human body when passing by, and determines the direction of people entering and exiting from the vertical direction and measures the width of the space occupied by the infrared rays emitted by the human body. The space width value is compared with the human shoulder width value preset in the single-chip microcomputer program to calculate the number of people entering and exiting. This method requires additional equipment for infrared monitoring, which increases costs, and the shoulder width between children and adults is greatly different, which is easy to count errors.

[0006] Patent document "A method for accurately monitoring the entry and exit of personnel in coal mines CN109948500B" discloses a method for accurately monitoring the entry and exit of personnel in coal mines, including the following steps: 1. Constructing an image processing system; 2. Storing facial images of staff members; 3. Training an AlexNet convolutional neural network model; 4. Monitoring the entry of personnel into the mining area; 5. Monitoring the exit of personnel from the mining area; 6. Monitoring the real-time personnel situation in the mining area. The present invention jointly monitors the entry and exit of personnel in the mining area through face comparison and head recognition of personnel, wherein the top of the personnel's head is identified and solved by the AlexNet convolutional neural network model, and the entry and exit of personnel in the mining area is intermittently counted to promptly detect non-operating personnel, thereby ensuring the standardization and safety of underground personnel operations. This method requires multiple cameras to jointly monitor to realize the entry and exit of personnel, which is highly complex, and the simple AlexNet convolutional neural network model has low accuracy, and head recognition is prone to false detection.

[0007] Patent document "People counting method, system and device based on face detection and recognition technology CN103793682B" discloses a people counting method, system and device based on face detection and recognition technology. The method segments N face images in the frame video and processes them into face data, which are stored as a new face data list; then the new face data is compared with the historical face data list line by line. If the similarity is less than the threshold, the new face data is added to the historical face data list, and the total number of people is increased by 1; then the next line is compared until the new face data list is compared. Then, the face data in the historical face data list that exceeds the valid time limit Tc is cleared, and the next video frame is processed until all the videos are processed. This invention is low-cost and easy and quick to deploy, but it can only be used in special occasions such as shopping malls, railway stations, and airport entrances and exits, and has poor applicability.

[0008] By analyzing the relevant patents in the prior art, the following shortcomings are found:

[0009] (1) Current methods for counting people mainly rely on infrared devices, which are applied to special scenarios such as single-person entry and exit, and are not accurate for counting multiple-person traffic;

[0010] (2) The personnel counting method for face detection and recognition has problems such as being time-consuming and having a single application scenario;

[0011] (3) In view of the abundant monitoring equipment, there is currently no targeted personnel counting method that relies on monitoring equipment. Summary of the invention

[0012] In view of the above problems, the present invention provides a method, device, equipment and medium for counting people entering and leaving the personnel for overcoming the above problems or at least partially solving the above problems.

[0013] The present invention provides the following scheme:

[0014] A method for counting people in and out, comprising:

[0015] Acquire a factory surveillance video, and obtain a plurality of consecutive frames of images to be processed from the factory surveillance video, wherein the plurality of frames of images to be processed all include head information of at least one person to be tested;

[0016] Inputting several frames of the to-be-processed pictures into a multi-target tracking model, the multi-target tracking model comprising a target detection network and a tracker; the target detection network is used to detect the head information contained in the several to-be-processed pictures to obtain several head bounding boxes; the tracker is used to track and output several prediction boxes using the several head bounding boxes;

[0017] Encode the plurality of prediction frames respectively to obtain a plurality of state dictionaries, each of the state dictionaries being used to record the state of the corresponding prediction frame through the vertical coordinate of the prediction frame and the relationship between the entry and exit lines;

[0018] For each person to be tested, an empty dictionary is initialized as an initial state dictionary, and the state dictionary corresponding to each person to be tested in the picture to be processed in the current frame is used as an end state dictionary and assigned to the initial state dictionary one by one;

[0019] Determine whether the states of the initial state dictionary and the final state dictionary having a common key have changed, wherein the key represents a corresponding target person, and the target person is one of the persons to be tested;

[0020] If it is determined that the state has not changed, it is determined that the target person has not crossed the entry and exit line, and the initial state dictionary remains unchanged;

[0021] If it is determined that the state has changed, it is determined that the target person has crossed the entry and exit line, then the end state dictionary is assigned to the initial state dictionary, and the state dictionary corresponding to the target person in the next frame of the image to be processed is used as a new end state dictionary;

[0022] The number of state changes of the initial state dictionary and the final state dictionary having a common key is counted to obtain a personnel entry and exit statistical result.

[0023] Preferably: if it is determined that the key of the state dictionary corresponding to the target person in the next frame of the image to be processed has changed relative to the key of the state dictionary corresponding to the target person in the current frame of the image to be processed, then the complement of the initial state dictionary to the ending state dictionary is selected to update the initial state dictionary.

[0024] Preferably: the format of the status dictionary is expressed as {id:[bbox,status], ..., id:[bbox,status]}, wherein id represents a key, bbox represents a prediction box, and status represents a status.

[0025] Preferably, the status status is obtained by calculating the following formula:

[0026]

[0027] Where: y is the ordinate of the prediction box, and door is the entry and exit line.

[0028] Preferably: the target detection network includes a DarkNet-53 backbone network and a C2f module, the neck of the target detection network uses a path-enhanced feature pyramid structure, and the prediction box of the target detection network adopts an Anchor-Free format.

[0029] Preferably: the tracker uses a bounding box obtained by a target detection network for tracking; the tracking process uses a Kalman filter to predict the bounding box and uses a Hungarian algorithm to match the target and the trajectory.

[0030] Preferably, the loss function used in the multi-target tracking model training is expressed by the following formula:

[0031]

[0032] Loss=Loss cls +γLoss reg

[0033] Where: γ represents the balance parameter, Loss cls Represents classification loss, Loss reg represents the regression loss, c iIndicates the label, p i represents the prediction score, f(.) represents the distance function, x i Represents the predicted coordinates, y i represents the label coordinates, and N represents the number of training batches.

[0034] A personnel entry and exit counting device, used to execute the above personnel entry and exit counting method, the device comprises:

[0035] A video acquisition unit, used to acquire a factory surveillance video, and to obtain a plurality of consecutive frames of images to be processed from the factory surveillance video, wherein each of the plurality of frames of images to be processed includes head information of at least one person to be tested;

[0036] A target tracking unit is used to input several frames of the to-be-processed pictures into a multi-target tracking model, wherein the multi-target tracking model includes a target detection network and a tracker; the target detection network is used to detect the head information contained in the several to-be-processed pictures to obtain several head bounding boxes; the tracker is used to track and output several prediction boxes using the several head bounding boxes;

[0037] A state dictionary encoding unit, used for respectively encoding the plurality of prediction frames to obtain a plurality of state dictionaries, each of the state dictionaries being used for recording the state of the corresponding prediction frame through the vertical coordinate of the prediction frame and the relationship between the entry and exit lines;

[0038] An initial state dictionary initial updating unit, used to initialize an empty dictionary as the initial state dictionary for each person to be tested, and to use the state dictionary corresponding to each person to be tested in the current frame of the to-be-processed picture as the end state dictionary and assign values ​​to the initial state dictionary one by one;

[0039] A state dictionary change judgment unit, used to judge whether the states of the initial state dictionary and the end state dictionary having a common key have changed, wherein the key represents a corresponding target person, and the target person is one of the persons to be tested;

[0040] An initial state dictionary holding unit, used to determine that if the state has not changed, then it is determined that the target person has not crossed the entry and exit line, and the initial state dictionary remains unchanged;

[0041] The initial state dictionary re-updating unit is used to determine that if the state changes, it is determined that the target person has crossed the entry and exit line, then the end state dictionary is assigned to the initial state dictionary, and the state dictionary corresponding to the target person in the next frame of the image to be processed is used as a new end state dictionary;

[0042] The result statistics unit is used to count the number of state changes of the initial state dictionary and the final state dictionary having a common key to obtain the personnel entry and exit statistics result.

[0043] A device for counting people entering and leaving the equipment, the device comprising a processor and a memory:

[0044] The memory is used to store program code and transmit the program code to the processor;

[0045] The processor is used to execute the above-mentioned personnel entry and exit counting method according to the instructions in the program code.

[0046] A computer-readable storage medium is used to store program codes, and the program codes are used to execute the above-mentioned personnel entry and exit statistics method.

[0047] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0048] The personnel entry and exit statistics method, device, equipment and medium provided in the embodiments of the present application are based on lightweight algorithms and can run efficiently on edge computing devices without relying on high-performance servers, thereby reducing the complexity and deployment cost of the system. Since the method has low requirements for hardware resources and is applicable to a variety of edge devices (such as smart cameras and embedded systems), the system is more convenient to install and expand in actual factory areas and adapt to scenarios of different scales.

[0049] This application provides an efficient personnel entry and exit identification and statistical algorithm, which can achieve a millisecond-level response speed and ensure real-time monitoring of personnel entry and exit at the factory gate. Through real-time data processing and rapid feedback, managers can instantly grasp the movements of personnel inside and outside the factory, which is convenient for improving safety management and responding to emergencies.

[0050] At the same time, this method reduces the requirements for hardware equipment and computing resources by optimizing the algorithm and system architecture, saving the cost of system construction. At the same time, since there is no need for complex installation and maintenance, the overall system is more economical and efficient. This method processes data through edge computing, reducing dependence on cloud resources, thereby further reducing data transmission costs and bandwidth requirements. It can effectively cope with crowded entry and exit scenes in complex environments, and through accurate analysis and identification of personnel behavior, it ensures high accuracy of statistical data and reduces statistical errors caused by false detection or missed detection. Its algorithm can adapt to a variety of environmental changes such as lighting and weather to ensure all-weather accuracy.

[0051] In addition, this method has wide applicability. It can not only meet the needs of counting the entry and exit of personnel at the factory gate, but can also be flexibly used in schools, communities, construction sites and other places that require personnel monitoring and management, showing excellent scalability and adaptability.

[0052] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0054] Figure 1 This is a flow chart of a personnel entry and exit statistics method provided by an embodiment of the present invention;

[0055] Figure 2 is a model training flow chart provided by an embodiment of the present invention;

[0056] Figure 3 is a structural block diagram of a target detection network provided by an embodiment of the present invention;

[0057] Figure 4 is a structural block diagram of a tracker provided by an embodiment of the present invention;

[0058] Figure 5 is a state dictionary update flow chart provided by an embodiment of the present invention;

[0059] Figure 6 is a personnel statistics flow chart provided by an embodiment of the present invention;

[0060] Figure 7 is a schematic diagram of a personnel entry and exit counting device provided by an embodiment of the present invention;

[0061] Figure 8 The figure is a schematic diagram of a personnel entry and exit counting device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. Based on the embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0063] See also Figure 1, is a personnel entry and exit statistics method provided by an embodiment of the present invention, such as Figure 1 As shown, the method may include:

[0064] S101: Obtain a factory surveillance video, and obtain a plurality of consecutive frames of images to be processed from the factory surveillance video, wherein the plurality of frames of images to be processed all include head information of at least one person to be tested;

[0065] S102: inputting several frames of the to-be-processed pictures into a multi-target tracking model, wherein the multi-target tracking model includes a target detection network and a tracker; the target detection network is used to detect the head information contained in the several to-be-processed pictures to obtain several head bounding boxes; the tracker is used to track and output several prediction boxes using the several head bounding boxes;

[0066] In specific implementation, the embodiment of the present application can provide that the target detection network includes a DarkNet-53 backbone network and a C2f module, the neck of the target detection network uses a path-enhanced feature pyramid structure, and the prediction box of the target detection network adopts an Anchor-Free format.

[0067] The tracker uses the bounding box obtained by the target detection network for tracking; the tracking process uses Kalman filtering to predict the bounding box and uses the Hungarian algorithm to match the target and the trajectory.

[0068] The loss function used in the multi-target tracking model training is expressed as follows:

[0069]

[0070]

[0071] Loss=Loss cls +γLoss reg

[0072] Where: γ represents the balance parameter, Loss cls Represents classification loss, Loss reg represents the regression loss, c i Indicates the label, p i represents the prediction score, f(.) represents the distance function, x i Represents the predicted coordinates, y i represents the label coordinates, and N represents the number of training batches.

[0073] S103: Encode several prediction boxes respectively to obtain several status dictionaries, each of which is used to record the status of the corresponding prediction box through the vertical coordinate of the prediction box and the relationship between the entry and exit lines; in specific implementation, the embodiment of the present application can provide the format of the status dictionary as {id:[bbox,status],...,id:[bbox,status]}, where id represents the key, bbox represents the prediction box, and status represents the status. The status status is calculated by the following formula:

[0074]

[0075] Where: y is the ordinate of the prediction box, and door is the entry and exit line.

[0076] S104: Initialize an empty dictionary as the initial state dictionary for each person to be tested, use the state dictionary corresponding to each person to be tested in the current frame of the image to be processed as the end state dictionary and assign values ​​to the initial state dictionary one by one;

[0077] S105: Determine whether the states of the initial state dictionary and the final state dictionary having a common key have changed, wherein the key represents a corresponding target person, and the target person is one of the persons to be tested;

[0078] S106: if it is determined that the state has not changed, it is determined that the target person has not crossed the entry and exit line, and the initial state dictionary remains unchanged;

[0079] S107: if it is determined that the state has changed, it is determined that the target person has crossed the entry and exit line, the end state dictionary is assigned to the initial state dictionary, and the state dictionary corresponding to the target person in the next frame of the image to be processed is used as a new end state dictionary;

[0080] During the tracking process, occlusion may occur, and the key of the person to be measured may change, resulting in the keys saved in the two state dictionaries not being completely consistent, and the state cannot be updated. In order to solve this problem, an embodiment of the present application can provide a method for determining that the key of the state dictionary corresponding to the target person in the image to be processed in the next frame has changed relative to the key of the state dictionary corresponding to the target person in the image to be processed in the current frame, and then selecting the complement of the initial state dictionary for the end state dictionary to update the initial state dictionary.

[0081] S108: Count the number of state changes of the initial state dictionary and the final state dictionary having a common key to obtain a personnel entry and exit statistics result.

[0082] The personnel entry and exit statistics method provided in the embodiment of the present application forms a complete technical process framework from constructing a model structure to organizing data sets for model training, and then to personnel statistics logic processing. The multi-target tracking model has the advantages of being lightweight, with both speed and accuracy. The training data set processes the open source data set and the self-annotated data set according to certain rules. Re-encode the tracker prediction results, build a state dictionary, standardize the data structure, and facilitate status query. Build a state dictionary update logic framework and personnel statistics rules. By constructing a state dictionary, this method only cares about the relative position relationship between the position of the person to be measured and the entry and exit line, and does not care about the trajectory, so it avoids the negative impact of conventional reliance on trajectory tracking to judge entry and exit, simplifies the personnel entry and exit judgment process, and improves the accuracy of personnel statistics.

[0083] The personnel entry and exit statistics method provided in the embodiment of the present application is introduced in detail below.

[0084] The lightweight and high-precision multi-target tracking model constructed in this method can provide real-time reasoning results in the tracking stage and can adapt to most scenarios. The constructed model can effectively extract edge information and semantic information, which improves the problem of easy loss of small target tracking.

[0085] A complete personnel statistics logic processing solution is proposed. This solution does not rely on the movement trajectory of the personnel to be measured, but is only related to the state change, which simplifies the judgment logic and improves the accuracy of statistics. At the same time, a state dictionary is proposed, and the recoding makes the data standardized, which provides convenience for the subsequent state update process.

[0086] This method abandons the conventional tracking of the entire person and uses head tracking, which can avoid the loss of the person to be measured in most cases due to occlusion under the monitoring perspective.

[0087] This application provides a method for accurately counting the number of people entering and leaving a factory based on deep learning, which includes the construction of a multi-target tracking model, the construction of a data set, and the logical processing of personnel statistics. The following are the main contents:

[0088] S1: Build a multi-target tracking model. The multi-target tracking model consists of two parts: target detection and tracker.

[0089] The performance of the multi-target tracking model determines the statistical effect, and target detection is the basis of the multi-target tracking algorithm, so training a high-quality target detection model is the key to the entire tracking process. First, build an advanced target detection model and tracker; second, organize the data set, download the open source target tracking data set, and annotate the videos you collected, and integrate the two data sets according to certain rules; then configure the relevant training environment; finally, start training the model. See the flowchart Figure 2 .

[0090] Furthermore, step S1 specifically includes the following contents:

[0091] S11: Build a multi-target tracking model.

[0092] (1) Build the target detection model. The backbone network uses an improved version of DarkNet-53, the neck adopts a path-enhanced feature pyramid structure, and the head adopts a decoupled head with no objectness branch and Anchor-Free format. The structure is shown in Figure 3 .

[0093] The target detection backbone network uses an improved version of DarkNet-53, replacing the C3 module with the C2f module to obtain richer gradient flow information. The backbone network maintains appropriate computational complexity while also having an ideal model depth, improving feature extraction capabilities. A path-enhanced feature pyramid structure is used in the neck, and features are fused from top to bottom and from bottom to top, deeply fusing the underlying edge information with the high-level semantic information. At the same time, a more advanced decoupling head is used to reduce the semantic conflicts between different source tasks, and the objectness branch is cancelled. The Anchor-Free format is used for the prediction box to reduce the computational complexity of the model.

[0094] (2) Build a tracker. The tracker uses the detection results based on target detection to track. That is, it creates a tracking trajectory and uses the tracking trajectory to match the target in each frame. The target is matched frame by frame. The structure is shown in Figure 4 .

[0095] The tracker used in this method is a tracking algorithm based on target detection. Like other non-ReID algorithms, it only uses the bounding box (bbox) obtained by target tracking for tracking. The tracking process uses Kalman filtering to predict the bounding box, and then uses the Hungarian algorithm to match the target and the trajectory.

[0096] S12: Build the detection model training and testing datasets.

[0097] Obtain surveillance videos and perform image screening, then annotate them, and convert the open source header dataset into the standard YOLO dataset format, and finally merge the two types of data.

[0098] In the specific implementation, obtain the factory monitoring video, use OpenCV to capture a picture every ten frames, and then filter out the pictures containing the head.

[0099] Download open source header datasets from the Internet to enrich scenarios and expand datasets.

[0100] Considering that tracking the entire person is prone to tracking failure after occlusion, this method uses the head for tracking to reduce occlusion scenarios. Use the labeling tool LabelImg to label the head of the person in the image and construct it into the standard YOLO format.

[0101] Preprocessing: First, expand all images by flipping left and right, mirroring, etc., then scale the images to 640, and finally divide each pixel by 255 for normalization.

[0102] S13: Configure the environment and train.

[0103] The main environment configuration is python3.6, torch1.8, cuda11.7. In the present invention, the pre-trained model trained on a large-scale data set is fine-tuned. The fine-tuning strategy is: using a logarithmic learning rate, the backbone network learning rate is reduced to one tenth of the original learning rate, and the output head learning rate remains unchanged; the backbone network is frozen for the first 20 epochs and unfrozen for the next 50 epochs. The output head updates the parameters normally during the entire training process. The Mosiac enhanced operation is turned off for the last 10 epochs. The loss function is shown in formulas (1), (2), and (3):

[0104]

[0105] Loss=Loss cls +γLoss reg (3)

[0106] Where: γ is the balance parameter, Loss cls Represents classification loss, Loss reg represents the regression loss, c i Indicates the label, p i represents the prediction score, f(.) represents the distance function, x i Represents the predicted coordinates, y i represents the label coordinates, and N represents the number of training batches.

[0107] S2: Personnel statistics logic processing.

[0108] S21: Build standardized data structures.

[0109] In order to facilitate status query, a unified data structure - status dictionary is constructed to re-encode the prediction box returned by the tracker. The dictionary format is {id:[bbox,status],...,id:[bbox,status]}. This dictionary records the status of the prediction box and is also called the status dictionary.

[0110] In order to build a unified data structure, the predicted box returned by the tracker is re-encoded to obtain the state dictionary. The encoding method is as shown in formula (4). The dictionary format is {id:[bbox,status], ..., id:[bbox,status]}. This dictionary records the status of the predicted box and is also called the state dictionary.

[0111]

[0112] Among them, y is the ordinate of the prediction box, and door is the entry and exit line.

[0113] S22: Update status.

[0114] Status update is the core of the personnel counting logic, using a more concise status change logic judgment to replace the conventional trajectory indication method. This process involves two status dictionaries: the initial status dictionary and the end status dictionary.

[0115] The initial state dictionary is updated for the first time to be an empty dictionary, and the update process is to assign the end state dictionary to the initial state dictionary.

[0116] For the subsequent update process of the initial state dictionary, the common key (that is, id, representing the number of prediction boxes) of the two state dictionaries is filtered out. If the states of the same id in the two state dictionaries are the same, it means that the person under test represented by the id has not crossed the entry and exit line, and the state remains unchanged; if the states are different, it means that the person under test has crossed the entry and exit line, and the state is updated. For the update of the end state dictionary, the state dictionary can be directly constructed using the tracking result of the next frame.

[0117] During the tracking process, there may be occlusions, and the ID of the person to be measured may change, resulting in the IDs saved in the two state dictionaries not being completely consistent, resulting in the state being unable to be updated. In view of this situation, the present invention proposes a solution. In this method, only the state in the state dictionary is retrieved. Therefore, for inconsistent IDs, the initial state dictionary is updated by selecting the complement of the end state dictionary for the initial state dictionary. For the specific process, see Figure 5 :

[0118] S23: Personnel statistics.

[0119] Compare the initial state dictionary and the final state dictionary. If the states are inconsistent, it is considered that the entry and exit lines are crossed. Statistics are collected based on the crossing states. The rules are as follows:

[0120] End state: out->initial state: in, then the person is considered to have gone out; end state: in->initial state: out, then the person is considered to have gone in. Finally, the number of state changes of the initial state dictionary and the end state dictionary with common keys is counted to obtain the personnel entry and exit statistics. Figure 6 .

[0121] In summary, the personnel entry and exit statistics method provided by this application is based on a lightweight algorithm and can run efficiently on edge computing devices without relying on high-performance servers, thereby reducing the complexity and deployment cost of the system. Since this method has low hardware resource requirements and is applicable to a variety of edge devices (such as smart cameras and embedded systems), the system is more convenient to install and expand in actual factory areas and adapt to scenarios of different scales.

[0122] This application provides an efficient personnel entry and exit identification and statistical algorithm, which can achieve a millisecond-level response speed and ensure real-time monitoring of personnel entry and exit at the factory gate. Through real-time data processing and rapid feedback, managers can instantly grasp the movements of personnel inside and outside the factory, which is convenient for improving safety management and responding to emergencies.

[0123] At the same time, this method reduces the requirements for hardware equipment and computing resources by optimizing the algorithm and system architecture, saving the cost of system construction. At the same time, since there is no need for complex installation and maintenance, the overall system is more economical and efficient. This method processes data through edge computing, reducing dependence on cloud resources, thereby further reducing data transmission costs and bandwidth requirements. It can effectively cope with crowded entry and exit scenes in complex environments, and through accurate analysis and identification of personnel behavior, it ensures high accuracy of statistical data and reduces statistical errors caused by false detection or missed detection. Its algorithm can adapt to a variety of environmental changes such as lighting and weather to ensure all-weather accuracy.

[0124] In addition, this method has wide applicability. It can not only meet the needs of counting the entry and exit of personnel at the factory gate, but can also be flexibly used in schools, communities, construction sites and other places that require personnel monitoring and management, showing excellent scalability and adaptability.

[0125] See also Figure 7 , the embodiment of the present application can also provide a personnel entry and exit counting device, such as Figure 7 As shown, the device for executing the above-mentioned personnel entry and exit counting method may include:

[0126] The video acquisition unit 701 is used to acquire a factory surveillance video, and obtain a plurality of consecutive frames of images to be processed from the factory surveillance video, wherein the plurality of frames of images to be processed all include head information of at least one person to be tested;

[0127] The target tracking unit 702 is used to input several frames of the to-be-processed pictures into a multi-target tracking model, wherein the multi-target tracking model includes a target detection network and a tracker; the target detection network is used to detect the head information contained in the several to-be-processed pictures to obtain several head bounding boxes; the tracker is used to track and output several prediction boxes using the several head bounding boxes;

[0128] A state dictionary encoding unit 703 is used to encode the plurality of prediction frames respectively to obtain a plurality of state dictionaries, each of which is used to record the state of the corresponding prediction frame through the vertical coordinate of the prediction frame and the relationship between the entry and exit lines;

[0129] The initial state dictionary initial updating unit 704 is used to initialize an empty dictionary as the initial state dictionary for each person to be tested, and to use the state dictionary corresponding to each person to be tested in the current frame of the to-be-processed picture as the end state dictionary and assign values ​​to the initial state dictionary one by one;

[0130] A state dictionary change judgment unit 705 is used to judge whether the states of the initial state dictionary and the end state dictionary having a common key have changed, wherein the key represents a corresponding target person, and the target person is one of the persons to be tested;

[0131] An initial state dictionary holding unit 706 is used to determine that if the state has not changed, then it is determined that the target person has not crossed the entry and exit line, and the initial state dictionary remains unchanged;

[0132] The initial state dictionary re-updating unit 707 is used to determine that if the state changes, it is determined that the target person has crossed the entry and exit line, then the end state dictionary is assigned to the initial state dictionary, and the state dictionary corresponding to the target person in the next frame of the image to be processed is used as a new end state dictionary;

[0133] The result statistics unit 708 is used to count the number of state changes of the initial state dictionary and the final state dictionary having a common key to obtain a personnel entry and exit statistics result.

[0134] The embodiment of the present application may also provide a personnel entry and exit counting device, the device comprising a processor and a memory:

[0135] The memory is used to store program code and transmit the program code to the processor;

[0136] The processor is used to execute the steps of the above-mentioned personnel entry and exit counting method according to the instructions in the program code.

[0137] like Figure 8As shown, a personnel entry and exit counting device provided in an embodiment of the present application may include: a processor 10, a memory 11, a communication interface 12 and a communication bus 13. The processor 10, the memory 11 and the communication interface 12 all communicate with each other through the communication bus 13.

[0138] In the embodiment of the present application, the processor 10 may be a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic devices, etc.

[0139] The processor 10 may call a program stored in the memory 11. Specifically, the processor 10 may execute operations in an embodiment of the personnel entry and exit counting method.

[0140] The memory 11 is used to store one or more programs, which may include program codes, and the program codes include computer operation instructions. In the embodiment of the present application, the memory 11 at least stores programs for implementing the following functions:

[0141] Acquire a factory surveillance video, and obtain a plurality of consecutive frames of images to be processed from the factory surveillance video, wherein the plurality of frames of images to be processed all include head information of at least one person to be tested;

[0142] Inputting several frames of the to-be-processed pictures into a multi-target tracking model, the multi-target tracking model comprising a target detection network and a tracker; the target detection network is used to detect the head information contained in the several to-be-processed pictures to obtain several head bounding boxes; the tracker is used to track and output several prediction boxes using the several head bounding boxes;

[0143] Encode the plurality of prediction frames respectively to obtain a plurality of state dictionaries, each of the state dictionaries being used to record the state of the corresponding prediction frame through the vertical coordinate of the prediction frame and the relationship between the entry and exit lines;

[0144] For each person to be tested, an empty dictionary is initialized as an initial state dictionary, and the state dictionary corresponding to each person to be tested in the picture to be processed in the current frame is used as an end state dictionary and assigned to the initial state dictionary one by one;

[0145] Determine whether the states of the initial state dictionary and the final state dictionary having a common key have changed, wherein the key represents a corresponding target person, and the target person is one of the persons to be tested;

[0146] If it is determined that the state has not changed, it is determined that the target person has not crossed the entry and exit line, and the initial state dictionary remains unchanged;

[0147] If it is determined that the state has changed, it is determined that the target person has crossed the entry and exit line, then the end state dictionary is assigned to the initial state dictionary, and the state dictionary corresponding to the target person in the next frame of the image to be processed is used as a new end state dictionary;

[0148] The number of state changes of the initial state dictionary and the final state dictionary having a common key is counted to obtain a personnel entry and exit statistical result.

[0149] In one possible implementation, the memory 11 may include a program storage area and a data storage area, wherein the program storage area can store an operating system and application programs required for at least one function (such as a file creation function, a data reading and writing function), etc.; the data storage area can store data created during use, such as initialization data, etc.

[0150] In addition, the memory 11 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0151] The communication interface 12 may be an interface of a communication module, and is used to connect to other devices or systems.

[0152] Of course, it should be noted that Figure 8 The structure shown does not constitute a limitation on the personnel entry and exit counting device in the embodiment of the present application. In actual applications, the personnel entry and exit counting device may include Figure 8 More or fewer components than shown, or combinations of certain components.

[0153] The embodiment of the present application may also provide a computer-readable storage medium, wherein the computer-readable storage medium is used to store program codes, and the program codes are used to execute the steps of the above-mentioned personnel entry and exit statistics method.

[0154] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0155] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0156] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0157] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method for counting people in and out, characterized in that: include: Acquire a factory surveillance video, and obtain a plurality of consecutive frames of images to be processed from the factory surveillance video, wherein the plurality of frames of images to be processed all include head information of at least one person to be tested; Inputting several frames of the to-be-processed pictures into a multi-target tracking model, the multi-target tracking model comprising a target detection network and a tracker; the target detection network is used to detect the head information contained in the several to-be-processed pictures to obtain several head bounding boxes; the tracker is used to track and output several prediction boxes using the several head bounding boxes; Encode the plurality of prediction frames respectively to obtain a plurality of state dictionaries, each of the state dictionaries being used to record the state of the corresponding prediction frame through the vertical coordinate of the prediction frame and the relationship between the entry and exit lines; For each person to be tested, an empty dictionary is initialized as an initial state dictionary, and the state dictionary corresponding to each person to be tested in the picture to be processed in the current frame is used as an end state dictionary and assigned to the initial state dictionary one by one; Determine whether the states of the initial state dictionary and the final state dictionary having a common key have changed, wherein the key represents a corresponding target person, and the target person is one of the persons to be tested; If it is determined that the state has not changed, it is determined that the target person has not crossed the entry and exit line, and the initial state dictionary remains unchanged; If it is determined that the state has changed, it is determined that the target person has crossed the entry and exit line, then the end state dictionary is assigned to the initial state dictionary, and the state dictionary corresponding to the target person in the next frame of the image to be processed is used as a new end state dictionary; The number of state changes of the initial state dictionary and the final state dictionary having a common key is counted to obtain a personnel entry and exit statistical result.

2. The personnel entry and exit counting method according to claim 1 is characterized in that: Determine that the key of the state dictionary corresponding to the target person in the next frame of the image to be processed changes relative to the key of the state dictionary corresponding to the target person in the current frame of the image to be processed, then select the complement of the initial state dictionary for the end state dictionary to update the initial state dictionary.

3. The personnel entry and exit counting method according to claim 1 is characterized in that: The format of the status dictionary is expressed as {id:[bbox,status], ..., id:[bbox,status]}, where id represents the key, bbox represents the prediction box, and status represents the status.

4. The personnel entry and exit counting method according to claim 3 is characterized in that: The status status is calculated by the following formula: Where: y is the ordinate of the prediction box, and door is the entry and exit line.

5. The personnel entry and exit counting method according to claim 1 is characterized in that: The target detection network includes a DarkNet-53 backbone network and a C2f module. The neck of the target detection network uses a path-enhanced feature pyramid structure. The prediction box of the target detection network adopts an Anchor-Free format.

6. The personnel entry and exit counting method according to claim 1, characterized in that: The tracker uses the bounding box obtained by the target detection network for tracking; the tracking process uses Kalman filtering to predict the bounding box and uses the Hungarian algorithm to match the target and the trajectory.

7. The personnel entry and exit counting method according to claim 1, characterized in that: The loss function used in the multi-target tracking model training is expressed as follows: Loss=Loss cls +γLoss reg Where: γ represents the balance parameter, Loss cls Represents classification loss, Loss reg represents the regression loss, c i Indicates the label, p i represents the prediction score, f(.) represents the distance function, x i Represents the predicted coordinates, y i represents the label coordinates, and N represents the number of training batches.

8. A device for counting people in and out, characterized in that: Used to execute the personnel entry and exit counting method according to any one of claims 1 to 7, the device comprises: A video acquisition unit, used to acquire a factory surveillance video, and to obtain a plurality of consecutive frames of images to be processed from the factory surveillance video, wherein each of the plurality of frames of images to be processed includes head information of at least one person to be tested; A target tracking unit is used to input several frames of the to-be-processed pictures into a multi-target tracking model, wherein the multi-target tracking model includes a target detection network and a tracker; the target detection network is used to detect the head information contained in the several to-be-processed pictures to obtain several head bounding boxes; the tracker is used to track and output several prediction boxes using the several head bounding boxes; A state dictionary encoding unit, used for respectively encoding the plurality of prediction frames to obtain a plurality of state dictionaries, each of the state dictionaries being used for recording the state of the corresponding prediction frame through the vertical coordinate of the prediction frame and the relationship between the entry and exit lines; An initial state dictionary initial updating unit, used to initialize an empty dictionary as the initial state dictionary for each person to be tested, and to use the state dictionary corresponding to each person to be tested in the current frame of the to-be-processed picture as the end state dictionary and assign values ​​to the initial state dictionary one by one; A state dictionary change judgment unit, used to judge whether the states of the initial state dictionary and the end state dictionary having a common key have changed, wherein the key represents a corresponding target person, and the target person is one of the persons to be tested; An initial state dictionary holding unit, used to determine that if the state has not changed, then it is determined that the target person has not crossed the entry and exit line, and the initial state dictionary remains unchanged; The initial state dictionary re-updating unit is used to determine that if the state changes, it is determined that the target person has crossed the entry and exit line, then the end state dictionary is assigned to the initial state dictionary, and the state dictionary corresponding to the target person in the next frame of the image to be processed is used as a new end state dictionary; The result statistics unit is used to count the number of state changes of the initial state dictionary and the final state dictionary having a common key to obtain the personnel entry and exit statistics result.

9. A personnel entry and exit counting device, characterized in that: The device comprises a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the personnel entry and exit counting method according to any one of claims 1 to 7 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program codes, and the program codes are used to execute the personnel entry and exit counting method according to any one of claims 1 to 7.

Images