Passenger flow statistics method, device, equipment and medium for multi-dimensional detection and tracking

Abstract

The present invention provides a passenger flow counting method, device, equipment, and medium for multi-dimensional detection and tracking. By acquiring a target scene video stream, frame images containing people are extracted; multi-dimensional detection of the head and torso of each frame image is performed based on a target detection model; the detected head object and torso object are tracked separately and assigned a tracking ID; the detection and tracking data of the head object and the detection and tracking data of the torso object corresponding to the same person object are associated and merged into the detection and tracking data of the person object, and the passenger flow object list is updated based on the detection and tracking data of the person object. On the basis of traditional single-dimensional passenger flow counting, by introducing multi-dimensional data detection and tracking, people can be accurately detected, and the robustness to interference from different dimensional detection results of the same person is greatly increased, thereby achieving very accurate passenger flow counting.

Description

Technical Field

[0001] The present invention relates to the technical field of image detection and image tracking, and in particular to a passenger flow counting method, device, equipment and medium for multi-dimensional detection and tracking. Background Art

[0002] Customer flow statistics technology is crucial for retail stores and large public venues. For example, retail stores can use customer flow statistics to analyze customer visit rates and retention rates, uncovering relationships between stores and providing crucial data support for operational decisions. Similarly, scenic spots can analyze customer flow patterns and the performance of individual attractions to understand visitor itineraries and provide data support for future development.

[0003] Current passenger flow statistics technologies are mainly divided into the following categories:

[0004] 1. Relying on hardware-based infrared passenger flow statistics, WiFi probe passenger flow statistics, etc. These technologies cannot track the trajectory, the passenger flow information is inaccurate, and two-way passenger flow statistics cannot be performed. They are now less used.

[0005] Second, traditional visual passenger counting based on manual feature extraction. This technology is based on video. For each frame, it detects a human face, head, head and shoulders, or human body using traditional manual feature extraction techniques. It then tracks these patterns to achieve passenger counting. This method can achieve trajectory tracking and bidirectional passenger counting, but due to limitations in detection technology, its accuracy is less than ideal.

[0006] Third, deep learning-based visual passenger counting technology. This technology is based on video. For each frame, it uses deep learning detection technology to detect one of the following patterns: faces, heads, head-and-shoulder patterns, and human bodies. It then tracks these patterns to achieve passenger counting. This method also enables trajectory tracking and two-way passenger counting, and detection accuracy has been significantly improved. However, accuracy is still limited due to factors such as trajectory interruptions and partial occlusion in the environment.

[0007] In summary, hardware-based passenger flow counting technology cannot cover all scenarios; visual passenger flow counting technology using traditional manual feature extraction is affected by detection technology and has low accuracy; deep learning-based visual passenger flow technology is affected by trajectory interruptions, partial occlusion in the environment, etc. during the tracking process, but its accuracy still reaches an ideal level. Summary of the Invention

[0008] In view of the above-mentioned shortcomings of the prior art, the purpose of the present invention is to provide a passenger flow counting method, device, equipment and medium for multi-dimensional detection and tracking, so as to solve the shortcomings of the passenger flow counting technology in the prior art.

[0009] To achieve the above-mentioned and other related purposes, the present invention provides a passenger flow counting method based on multi-dimensional detection and tracking, the method comprising: acquiring a target scene video stream and extracting frame images in which people are present; performing multi-dimensional detection of the head and torso of each frame image based on a target detection model; tracking the detected head object and torso object separately and assigning a tracking ID; associating the detection and tracking data of the head object and the detection and tracking data of the torso object corresponding to the same person object and fusing them into the detection and tracking data of the person object, and updating the passenger flow object list based on the detection and tracking data of the person object.

[0010] In one embodiment of the present invention, the method for obtaining the target detection model includes: obtaining a surveillance video of the target scene and extracting frame images in which people are present; separately labeling the head and torso of each person in the frame image; dividing the labeled frame images into training data and evaluation data, training the training data based on a deep learning visual object detection method, and verifying it using the evaluation data to obtain a trained target detection model.

[0011] In one embodiment of the present invention, the detection and tracking data of the head object and the detection and tracking data of the body object corresponding to the same person object are associated and merged into the detection and tracking data of the person object, including: traversing the body object list, and searching for all unassociated head objects whose detection frame areas overlap with the current body object by more than a preset percentage: binding the found head object and the current body object into a person object to store it in the person object list, and marking the head object as associated; or, when the head object is not found, making the current body object a separate person object and storing it in the person object list.

[0012] In one embodiment of the present invention, the step of binding the found head object and the current body object into a person object to store them in a person object list, and marking the head object as associated, includes: when a head object is found, binding the head object and the current body object into a person object to store them in a person object list, and marking the head object as associated; or, when multiple head objects are found, searching for the head object closest to the center point of the current body object, binding the head object and the current body object into a person object to store them in a person object list, and marking the selected head object as associated.

[0013] In one embodiment of the present invention, the detection and tracking data of the head object and the detection and tracking data of the body object corresponding to the same person object are associated and merged into the detection and tracking data of the person object, including: after traversing the body object list, traversing the head object list; judging in turn whether the current head object is marked as associated; if so, skipping the current head object and judging the next head object; if not, the current head object becomes a person object alone and is stored in the person object list.

[0014] In one embodiment of the present invention, updating the passenger flow object list based on the detection and tracking data of the personnel object includes: traversing the passenger flow object list; determining whether there is an identical head tracking ID or body tracking ID in the personnel object list; if so, updating the status of the personnel object to the current passenger flow object and marking the personnel object as updated; if not, determining whether the passenger flow object has timed out; if so, recording the passenger flow and deleting the passenger flow object.

[0015] In one embodiment of the present invention, the updating of the passenger flow object list based on the detection and tracking data of the personnel object includes: after traversing the passenger flow object list, traversing the personnel object list; determining whether the personnel object has been updated; if so, skipping the object; if not, creating a new passenger flow object for the personnel object and adding it to the passenger flow object list group; and obtaining the updated passenger flow object list.

[0016] To achieve the above-mentioned and other related purposes, the present invention provides a passenger flow counting device with multi-dimensional detection and tracking, the device comprising: an extraction module for acquiring a target scene video stream and extracting frame images containing people; a detection module for performing multi-dimensional detection of the head and torso of each frame image based on a target detection model; a tracking module for tracking the detected head objects and torso objects respectively and assigning tracking IDs; and a fusion module for associating the detection and tracking data of the head object and the detection and tracking data of the torso object corresponding to the same person object and fusing them into the detection and tracking data of the person object, and updating the passenger flow object list based on the detection and tracking data of the person object.

[0017] To achieve the above-mentioned object and other related objects, the present invention provides a computer device, which includes: a memory and a processor; the memory stores a computer program, and the processor executes the computer program to implement the above-mentioned method.

[0018] To achieve the above-mentioned object and other related objects, the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned method when executed by a processor.

[0019] As described above, the present invention provides a passenger flow statistics method, apparatus, device and medium for multi-dimensional detection and tracking, which obtains a target scene video stream and extracts frame images in which people are present; performs multi-dimensional detection of the head and torso on each frame image based on a target detection model; tracks the detected head object and torso object separately and assigns a tracking ID; associates the detection and tracking data of the head object and the detection and tracking data of the torso object corresponding to the same person object and fuses them into the detection and tracking data of the person object, and updates the passenger flow object list based on the detection and tracking data of the person object.

[0020] It has the following beneficial effects:

[0021] Based on traditional single-dimensional passenger flow statistics, the present invention introduces multi-dimensional data detection and tracking, which can accurately detect people and greatly increase the robustness to interference from different dimensional detection results of the same person, thereby achieving very accurate passenger flow statistics. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 Shown is a flow chart of a passenger flow statistics method with multi-dimensional detection and tracking in one embodiment of the present invention.

[0023] Figure 2 Shown is a schematic diagram of the association and fusion process in one embodiment of the present invention.

[0024] Figure 3 Shown is a schematic diagram of the process of updating the passenger flow object list in one embodiment of the present invention.

[0025] Figure 4 Shown is a module diagram of a passenger flow counting device with multi-dimensional detection and tracking in one embodiment of the present invention.

[0026] Figure 5 Shown is a schematic structural diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0027] The following describes the embodiments of the present invention through specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0028] It should be noted that the illustrations provided in the following embodiments are merely schematic illustrations of the basic concept of the present invention. Therefore, the illustrations only show components related to the present invention and are not drawn according to the number, shape, and size of components in actual implementation. In actual implementation, the type, quantity, and proportion of each component may be changed arbitrarily, and the component layout may also be more complex.

[0029] Current passenger flow counting technology has the following shortcomings: hardware-based passenger flow counting technology cannot cover all scenarios; visual passenger flow counting technology that uses traditional manual feature extraction is affected by detection technology and has low accuracy; deep learning-based visual passenger flow technology is affected by trajectory interruptions, partial occlusion in the environment, etc. during the tracking process, but its accuracy still reaches an ideal level.

[0030] To solve the above problems, the present invention uses an intelligent detection algorithm based on deep learning technology. During detection, it detects data of multiple dimensions at the same time. For example, taking the head and torso as an example, it detects the head object and the torso object at the same time. During tracking, it tracks the detection objects of different dimensions at the same time, and then fuses these objects. The detection results of different dimensions of the same person are grouped together to obtain a list of personnel objects. For the same person, as long as the detection object of a certain dimension is maintained in the tracking state, even if the detection and tracking of other dimensions is lost, it does not affect the tracking state of the person. This greatly improves the accuracy of detection and tracking. Due to the use of multi-dimensional detection, this technology can accurately detect people, and has good robustness to the interference of detection results of different dimensions of the same person, thereby achieving very accurate passenger flow statistics.

[0031] like Figure 1 FIG. 1 is a flow chart showing a passenger flow counting method for multi-dimensional detection and tracking in one embodiment of the present invention. As shown in the figure, the method includes:

[0032] Step S101: Acquire a target scene video stream and extract frames where people are present.

[0033] Simply put, the video data collected by the camera in the target scene is taken as a video stream, and then extracted frame by frame to extract the frame image containing people or where there are people.

[0034] For example, target scenarios include retail stores or large public venues, where customer flow statistics and analysis are required. For example, retail stores can analyze customer visit rates and retention rates through customer flow statistics, exploring store relationships and providing important data support for operational decisions. Similarly, scenic spots can analyze customer flow patterns and the conditions of various attractions to understand visitor itineraries and provide data support for later development.

[0035] Step S102: Perform multi-dimensional detection of the head and body of each frame image based on the target detection model.

[0036] In this application, the method for obtaining the target detection model includes:

[0037] A. Obtain surveillance videos of the target scene and extract frame images containing people. For example, collect some surveillance videos of the target scene and extract frames containing people and save them as images.

[0038] B. Labeling the head and torso of each person in the frame image, such as manually labeling the head and body parts of the extracted image.

[0039] C. Divide the annotated frame images into training data and evaluation data, train the training data based on the deep learning visual object detection method, and use the evaluation data for verification to obtain a trained object detection model.

[0040] Preferably, the labeled data is divided into training data and evaluation data, and the target detection model is trained on the training data using a suitable deep learning visual object detection method, and verified on the evaluation data, ultimately obtaining a usable target detection model with a relatively balanced detection rate and accuracy in each dimension.

[0041] Simply put, in this invention, the definition of a person is no longer limited to just the face or body, but rather encompasses a multi-dimensional complex. In this application, we primarily perform detection, tracking, and fusion of head and torso data. However, this invention is not limited to these specific components.

[0042] Preferably, the head includes: any one of the human head as a whole, the face part, the back of the head part, and the head and shoulders part; the body includes: any one of the body as a whole, the trunk part, the upper limb part, the lower limb part, the upper body part, and the lower body part.

[0043] Preferably, in some scenarios requiring higher detection accuracy, the present invention is not limited to detection in two dimensions of head and body, but can also be divided into multiple dimensions for detection.

[0044] For example, the present invention can also detect three dimensions: head, upper body, and lower body; or four dimensions: face, torso, upper limbs, and lower limbs. Similarly, the present invention can also detect multiple dimensions, not just two.

[0045] The advantage of performing multi-dimensional detection is that when one dimension is lost, tracking can still be maintained by other dimensions, thus not affecting the tracking detection of a certain person, which can greatly improve the accuracy of detection and tracking.

[0046] In addition, yolov5 is used in the present invention to detect multi-dimensional objects, but the present invention is not limited to this detection method. A variety of deep learning detection methods can also be used to detect dimensional objects such as faces, heads, heads and shoulders, and human bodies in the present invention, such as Faster RCNN, SSD and other detection algorithms. These deep learning detection methods are all covered within the scope of protection of the present invention.

[0047] Step S103: Track the detected head object and body object respectively and assign tracking IDs.

[0048] Simply put, each dimension of the detection object is tracked independently and assigned a tracking ID. Specifically, the independent tracking of multi-dimensional detection objects is achieved through camera installation and debugging.

[0049] For example, when initializing the tracking module, a corresponding number of tracking modules are created according to the number of dimensions. Different tracking modules process detection results of different dimensions.

[0050] It should be noted that there are also multiple tracking methods for tracking these objects. Preferably, the SORT tracking method is used in the present invention, but the present invention is not limited to this method. Other tracking algorithms such as DeepSORT and SiamR-CNN can also replace the tracking method in this solution. These tracking algorithms are all included in the scope of protection of the present invention.

[0051] Step S104: Correlate the detection and tracking data of the head object and the detection and tracking data of the body object corresponding to the same person object and fuse them into the detection and tracking data of the person object, and update the passenger flow object list according to the detection and tracking data of the person object.

[0052] In this application, the detection and tracking data includes: an object list and a tracking ID.

[0053] Preferably, a multi-dimensional detection of the head and body is performed on each frame image based on the target detection model to obtain a head object list and a body object list respectively.

[0054] Preferably, the detected head object and body object are tracked and assigned tracking IDs respectively, to obtain a head object tracking ID and a body object tracking ID respectively.

[0055] Preferably, the detection and tracking data integrated into the person object includes a person object list and a person object tracking ID.

[0056] In one embodiment of the present application, associating the detection and tracking data of the head object and the detection and tracking data of the torso object corresponding to the same person object and fusing them into the detection and tracking data of the person object includes:

[0057] A. Traverse the list of body objects and search for all unassociated head objects whose detection box overlaps with the current body object by more than a preset percentage.

[0058] In some examples, a detected head and torso object will display a rectangular detection frame. The rectangular detection frames corresponding to the head and torso objects are similar in size. Therefore, when the head and torso detection frames overlap to a certain degree, it can be determined that the same person has been detected. In the present invention, overlap refers to the degree of area overlap, or IOU (Intersection over Union). Preferably, the present invention sets the preset percentage to 90%. This means that the overlapping area of ​​the head and torso object accounts for at least 90% of the area of ​​the head detection frame.

[0059] B. Bind the found head object and the current body object into a person object to store in the person object list, and mark the head object as associated; or, when the head object is not found, make the current body object a separate person object and store in the person object list.

[0060] There are two cases when the head object is found:

[0061] When a head object is found, the head object is bound to the current body object as a person object to be stored in the person object list, and the head object is marked as associated.

[0062] Alternatively, when multiple head objects are found, the head object closest to the center point of the current body object is found, the head object and the current body object are bound to a person object to store in the person object list, and the selected head object is marked as associated.

[0063] In some examples, when multiple head objects are found, they are displayed as multiple rectangular detection boxes. At this time, we compare which head object's detection box is closer to the current body object's detection box. Usually, we can search and compare along the horizontal direction or along the X-axis of the coordinate system.

[0064] C. After traversing the body object list, traverse the head object list;

[0065] D. Determine in turn whether the current head object is marked as associated; if so, skip the current head object and determine the next head object; if not, the current head object becomes a separate person object and is stored in the person object list.

[0066] like Figure 2As shown in FIG, a flowchart of association and fusion in one embodiment of the present invention is shown. Here, Head refers to the head object, Body refers to the body object, and Person refers to the person object. As shown in the figure, the steps are as follows:

[0067] 1) The input data is the multi-dimensional detection and tracking results, namely the head list and body list and their respective tracking IDs.

[0068] 2) Iterate over the list of Body objects.

[0069] 2.1) For the current Body object, find all unassociated Head objects whose detection box overlaps with its area by more than 90%.

[0070] 2.2) If one is found, the Head object and the Body object are bound to form a Person object and stored in the Person object list and the Head object is marked as associated;

[0071] 2.3) If multiple Head objects are found, search for the Head object closest to the center point of the current Body object along the X-axis or horizontal direction, bind the two together, and create a Person object. This Person object is stored in the Person list, and the selected Head object is marked as associated.

[0072] 2.4) If no Head object is found, the Body becomes a Person and is stored in the Person list.

[0073] 3) Traverse the list of Head objects.

[0074] 3.1) If the Head object has been associated, skip this Head object and process the next Head object.

[0075] 3.2) If the Head object is not associated, the Head becomes a Person on its own and is stored in the Person list.

[0076] In one embodiment of the present application, updating the passenger flow object list based on the detection and tracking data of the person object includes:

[0077] A. Traverse the passenger flow object list;

[0078] B. Determine whether there is the same head tracking ID or body tracking ID in the person object list;

[0079] C. If yes, update the status of the person object to the current passenger flow object and mark the person object as updated; if not, determine whether the passenger flow object has timed out;

[0080] D. If the timeout has expired, the passenger flow is recorded and the passenger flow object is deleted. Preferably, the timeout is determined if the last 30 frames have not been updated.

[0081] E. After traversing the passenger flow object list, traverse the personnel object list;

[0082] F. Determine whether the personnel object has been updated;

[0083] G. If yes, skip the object; if no, create a new passenger flow object for the person object and add it to the passenger flow object list group;

[0084] H. Obtain the updated passenger flow object list.

[0085] like Figure 3 As shown in FIG, a flow chart of updating a passenger flow object list in one embodiment of the present invention is shown. Here, Head refers to a head object, Body refers to a body object, and Person refers to a person object. As shown in the figure, the steps are as follows:

[0086] 1) The input data is the fused list of Person objects and the list of passenger flow objects.

[0087] 2) Traverse the passenger flow objects.

[0088] 2.1) If there is the same Head tracking ID or the same Body tracking ID in the Person object list, the status of the Person object is updated to the current passenger flow object and the Person object is marked as updated.

[0089] 2.2) If there is no identical Head Tracking ID or Body Tracking ID in the Person object list, determine whether the passenger flow object has timed out. If so, record the passenger flow and delete the passenger flow object. Preferably, the timeout is determined based on the absence of updates in the last 30 frames.

[0090] 3) Iterate over the list of Person objects.

[0091] 3.1) If the Person object has been marked as updated, skip the object.

[0092] 3.2) If the Person object has not been updated, it means it is a new customer flow object. Create a new customer flow object for the Person object and add it to the customer flow object list group.

[0093] 4) Finally, obtain the updated passenger flow object list.

[0094] 5) Based on the passenger flow object list, you can obtain the passenger flow results. If you need to continue processing, receive the next frame in the video stream and repeat the steps.

[0095] Generally speaking, traditional methods are based on single-dimensional passenger flow statistics. For example, when a guest is still in the scene but their head is in the shadow or blocked, the head cannot be detected. This will cause the trajectory to be interrupted, and the passenger flow counter will be incorrectly increased by 1, resulting in inaccurate passenger flow data.

[0096] The present invention detects data of multiple dimensions simultaneously during detection. For example, taking the head and torso as an example, after adopting multi-dimensional passenger flow statistics, even if the guest's head cannot be detected, the guest's body parts can still be detected and effectively tracked, and the guest is still in the correct state. This ensures the robustness and accuracy of passenger flow statistics, thereby making the passenger flow statistics data more valuable.

[0097] Based on traditional single-dimensional passenger flow statistics, the present invention introduces multi-dimensional data detection and tracking, which can accurately detect people and greatly increase the robustness to interference from different dimensional detection results of the same person, thereby achieving very accurate passenger flow statistics.

[0098] like Figure 4 FIG. 1 is a block diagram showing a passenger flow counting device for multi-dimensional detection and tracking according to an embodiment of the present invention. As shown in the figure, the passenger flow counting device for multi-dimensional detection and tracking 400 includes:

[0099] Extraction module 401, used to obtain the target scene video stream and extract the frame image with the presence of people;

[0100] Detection module 402, configured to perform multi-dimensional detection of the head and body of each frame image based on the target detection model;

[0101] Tracking module 403, used to track the detected head object and body object respectively and assign tracking IDs;

[0102] The fusion module 404 is used to associate the detection and tracking data of the head object and the detection and tracking data of the body object corresponding to the same person object and fuse them into the detection and tracking data of the person object, and update the passenger flow object list according to the detection and tracking data of the person object.

[0103] It is understandable that the device 400 can achieve the following through the operation of each module: Figure 1 The passenger flow statistics method of multi-dimensional detection and tracking.

[0104] It should be noted that the division of the various modules of the above device is merely a division of logical functions. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can all be implemented in the form of software called by a processing element; or all be implemented in the form of hardware; or some modules can be implemented in the form of software called by a processing element, and some modules can be implemented in the form of hardware. For example, fusion module 404 can be a separate processing element, or it can be integrated into a chip of the above device. In addition, it can be stored in the form of program code in the memory of the above device, and called by a processing element of the above device to perform the functions of the above fusion module 404. The implementation of other modules is similar. In addition, these modules can all or partly be integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. During implementation, each step of the above method or each of the above modules can be completed by hardware integrated logic circuits in the processor element or by software instructions.

[0105] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented by scheduling program code through a processing element, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0106] like Figure 5 FIG. 5 is a block diagram of a computer device according to an embodiment of the present invention. As shown in the figure, the computer device 500 includes a memory 501 and a processor 502. The memory 501 stores a computer program, and the processor 502 executes the computer program to implement the following. Figure 1 The passenger flow statistics method of multi-dimensional detection and tracking.

[0107] The memory 501 may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0108] The processor 502 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0109] In one embodiment of the present invention, a computer-readable storage medium stores a data transmission program, which is executed by a processor to implement the following Figure 1 The passenger flow statistics method of multi-dimensional detection and tracking.

[0110] Regarding the computer-readable storage medium, those skilled in the art will appreciate that all or part of the steps in the aforementioned method embodiments can be implemented using hardware associated with the computer program. The aforementioned image processing program can be stored in a computer-readable storage medium. When executed, the program performs the steps in the aforementioned method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0111] These computer programs can also be loaded onto a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide the instructions for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0112] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0113] In summary, the present invention provides a passenger flow statistics method, device, equipment and medium for multi-dimensional detection and tracking, which can effectively overcome the various shortcomings of the existing technology and has high industrial utilization value.

[0114] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the present invention. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical principles disclosed herein are intended to be covered by the claims of the present invention.

Claims

1. A passenger flow statistics method based on multi-dimensional detection and tracking, characterized in that: The method comprises: Obtain the target scene video stream and extract the frame images where people are present; Perform multi-dimensional detection of the head and body of each frame image based on the target detection model; Track the detected head object and body object separately and assign tracking IDs; Correlating the detection and tracking data of the head object and the detection and tracking data of the torso object corresponding to the same person object and fusing them into the detection and tracking data of the person object, and updating the passenger flow object list based on the detection and tracking data of the person object; The associating the detection and tracking data of the head object and the detection and tracking data of the body object corresponding to the same person object and fusing them into the detection and tracking data of the person object includes: Traverse the list of body objects and find all unassociated head objects whose detection box overlaps with the current body object by more than a preset percentage: Binding the found head object and the current body object into a person object, storing the object in a person object list, and marking the head object as associated; or, if no head object is found, binding the current body object into a separate person object and storing the object in a person object list, wherein the person object list includes at least one person object and a corresponding head tracking ID and body tracking ID, or a body tracking ID; The updating of the passenger flow object list based on the detection and tracking data of the personnel object includes: Traverse the passenger flow object list; Determine whether there is a head tracking ID or a body tracking ID in the person object list that is identical to that in the passenger flow object list; If so, the status of the person object is updated to the current passenger flow object and the person object is marked as updated; if not, it is determined whether the passenger flow object has timed out, and the status of the person object refers to the status of the person object in the target scene; If the timeout has expired, the passenger flow will be recorded and the passenger flow object will be deleted; Iterate through the list of person objects; Determine whether the person object has been updated; If yes, skip the person object; if no, create a new passenger flow object for the person object and add it to the passenger flow object list; Get the updated passenger flow object list.

2. The passenger flow counting method of multi-dimensional detection and tracking according to claim 1 is characterized in that: The method for acquiring the target detection model includes: Obtain surveillance video of the target scene and extract frame images where people are present; Marking the head and torso of each person in the frame image respectively; The labeled frame images are divided into training data and evaluation data, the training data is trained based on the deep learning visual object detection method, and the evaluation data is used for verification to obtain a trained target detection model.

3. The passenger flow counting method of multi-dimensional detection and tracking according to claim 1 is characterized in that: The step of binding the found head object and the current body object into a person object to store the person object in a list of person objects, and marking the head object as associated, includes: When a head object is found, the head object is bound to the current body object as a person object to be stored in the person object list, and the head object is marked as associated; Alternatively, when multiple head objects are found, the head object closest to the center point of the current body object is found, the head object and the current body object are bound to a person object to store in the person object list, and the selected head object is marked as associated.

4. The passenger flow counting method of multi-dimensional detection and tracking according to claim 1 is characterized in that: The associating the detection and tracking data of the head object and the detection and tracking data of the body object corresponding to the same person object and fusing them into the detection and tracking data of the person object includes: After traversing the body object list, traverse the head object list; Determine in turn whether the current head object is marked as associated; If so, skip the current head object and determine the next head object; if not, the current head object becomes a separate person object and is stored in the person object list.

5. A passenger flow counting device with multi-dimensional detection and tracking, characterized in that: The device comprises: The extraction module is used to obtain the target scene video stream and extract the frame images where people are present; The detection module is used to perform multi-dimensional detection of the head and body of each frame image based on the target detection model; A tracking module, configured to track the detected head object and body object respectively and assign tracking IDs; A fusion module is used to associate the detection and tracking data of the head object and the detection and tracking data of the body object corresponding to the same person object and fuse them into the detection and tracking data of the person object, and update the passenger flow object list based on the detection and tracking data of the person object; Specifically, the fusion module is used to traverse the body object list and search for all unassociated head objects whose detection box area overlaps with the current body object by more than a preset percentage; bind the found head object and the current body object into a person object to store in the person object list, and mark the head object as associated; or, when the head object is not found, the current body object is separately converted into a person object and stored in the person object list, the person object list includes at least one person object and the corresponding head tracking ID and body tracking ID, or body tracking ID; traverse the passenger flow object list; determine the person Whether there is a head tracking ID or body tracking ID in the object list that is the same as that in the passenger flow object list; if so, update the status of the person object to the current passenger flow object and mark the person object as updated; if not, determine whether the passenger flow object has timed out, and the status of the person object refers to the status of the person object in the target scene; if it has timed out, record the passenger flow and delete the passenger flow object; traverse the personnel object list; determine whether the person object has been updated; if so, skip the person object; if not, create a new passenger flow object for the person object and add it to the passenger flow object list; obtain the updated passenger flow object list.

6. A computer device, characterized in that: The device includes: a memory and a processor; the memory stores a computer program, and the processor executes the computer program to implement the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images