Passenger flow statistics method, device and computer equipment based on pedestrians holding umbrellas

Abstract

The present application relates to a passenger flow counting method, device and computer equipment based on pedestrians holding umbrellas, wherein the passenger flow counting method based on pedestrians holding umbrellas includes: detecting each input image through a well-trained target detection model to obtain a corresponding target label; the target label includes an umbrella label; tracking pedestrian targets in multiple consecutive frames of input images based on different target labels; determining the target label and corresponding identity identifier that match each pedestrian target; further, based on the movement trajectory information corresponding to each identity identifier, passenger flow counting is performed for each pedestrian target. Through this application, the problem of being unable to accurately count pedestrian flow when pedestrians hold umbrellas is solved, and accurate pedestrian flow counting when pedestrians hold umbrellas is achieved.

Description

Technical Field

[0001] The present application relates to the field of video surveillance technology, and in particular to a method, device and computer equipment for counting passenger flow based on pedestrians holding umbrellas. Background Art

[0002] Passenger flow statistics are designed to count and analyze the pedestrian flow at a certain location or venue, helping users obtain the distribution of people at different time periods and locations, and promptly discover whether there are areas with abnormally dense crowds, etc., in order to facilitate the implementation of safety management.

[0003] In the existing passenger flow counting method, the position of the pedestrian head area is detected and tracked, and the number of people is counted based on the detection and tracking results. However, in the scenario where pedestrians enter or exit with umbrellas, the above method is affected by the umbrellas and cannot accurately count the pedestrian flow in the case of pedestrians holding umbrellas.

[0004] With regard to the problem in related technologies that pedestrian flow cannot be accurately counted when pedestrians are holding umbrellas, no effective solution has been proposed so far. Summary of the invention

[0005] In this embodiment, a method, device and computer equipment for counting passenger flow based on pedestrians holding umbrellas are provided to solve the problem in the related art that it is impossible to accurately count the pedestrian flow when pedestrians hold umbrellas.

[0006] In a first aspect, a passenger flow counting method based on pedestrians holding umbrellas is provided in this embodiment, and the method includes:

[0007] Each input image is detected by a trained target detection model to obtain a corresponding target label; the target label includes an umbrella label;

[0008] Tracking pedestrian targets in the input images of multiple consecutive frames based on different target labels;

[0009] Determining the target label and the corresponding identity identifier that matches each of the pedestrian targets;

[0010] Based on the movement trajectory information corresponding to each of the identity tags, passenger flow statistics are performed on each of the pedestrian targets.

[0011] In some embodiments, when each of the target tags is an umbrella tag, determining the target tag and the corresponding identity identifier that matches each of the pedestrian targets includes:

[0012] Determining the umbrella label that matches each of the pedestrian targets;

[0013] The first identifier of the matched umbrella tag is used as the identity identifier of the pedestrian target; the first identifier is used to uniquely identify the corresponding umbrella tag.

[0014] In some embodiments, when the target tag further includes a pedestrian head and shoulder tag and a pedestrian holding an umbrella tag, determining the target tag and the corresponding identity identifier that match each pedestrian target includes:

[0015] Determine a plurality of target tags that match each of the pedestrian targets; wherein there is an inclusion relationship between the position areas corresponding to the target tags;

[0016] Based on the target tags corresponding to each of the pedestrian targets, an identity of the pedestrian target is determined.

[0017] In some embodiments, when the target tag further includes a pedestrian head and shoulder tag and a pedestrian holding an umbrella tag, determining the target tag and the corresponding identity identifier that match each pedestrian target further includes:

[0018] Determine a plurality of target tags that match each of the pedestrian targets; wherein there is an inclusion relationship between the position areas corresponding to the target tags, and the position movement trajectories corresponding to the target tags satisfy a preset similarity relationship between each other;

[0019] Based on the target tags corresponding to each of the pedestrian targets, an identity of the pedestrian target is determined.

[0020] In some embodiments, determining the identity of the pedestrian target based on each target tag corresponding to each pedestrian target includes:

[0021] Determining a plurality of target tags matching each of the pedestrian targets;

[0022] Acquire a second identifier of the target tag with the highest priority; the second identifier is used to uniquely identify the corresponding target tag;

[0023] The second identifier is used as the identity identifier of the pedestrian target.

[0024] In some of the embodiments, determining whether there is an inclusion relationship between the location areas corresponding to the target tags includes:

[0025] Obtaining the position coordinates of each target tag; the position coordinates include the coordinates of two diagonal vertices of the rectangular detection box corresponding to the target tag;

[0026] Calculating the position coordinates of the target tags of different categories to obtain the inclusion ratio between the position areas corresponding to the target tags;

[0027] When the inclusion ratio is greater than a preset threshold, it is determined that there is an inclusion relationship between the target tags.

[0028] In some of the embodiments, the passenger flow statistics of each of the pedestrian targets based on the movement trajectory information corresponding to each of the identity identifiers include:

[0029] Based on the movement trajectory information corresponding to each of the identity identifiers and the preset tripwire rule, determining the entry and exit status of the pedestrian target corresponding to the identity identifier;

[0030] Based on the entry and exit status of each of the pedestrian targets, passenger flow statistics are performed on each of the pedestrian targets.

[0031] In a second aspect, a passenger flow counting device based on umbrella-holding pedestrians is provided in this embodiment, and the device includes: a detection module, a tracking module, a matching module and a counting module;

[0032] The detection module is used to detect each input image by training a complete target detection model to obtain a corresponding target label; the target label includes an umbrella label;

[0033] The tracking module is used to track pedestrian targets in the input images of multiple consecutive frames based on different target labels;

[0034] The matching module is used to determine the target label and the corresponding identity identifier that matches each pedestrian target;

[0035] The statistical module is used to perform passenger flow statistics on each of the pedestrian targets based on the movement trajectory information corresponding to each of the identity identifiers.

[0036] In a third aspect, a computer device is provided in this embodiment, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the passenger flow counting method based on umbrella-holding pedestrians described in the first aspect is implemented.

[0037] In a fourth aspect, in this embodiment, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the passenger flow counting method based on umbrella-holding pedestrians described in the first aspect is implemented.

[0038] Compared with the related art, the passenger flow counting method, device and computer equipment based on umbrella-holding pedestrians provided in this embodiment detect each input image by training a complete target detection model to obtain a corresponding target label; the target label includes an umbrella label; based on different target labels, pedestrian targets in multiple consecutive frames of input images are tracked; the target label and the corresponding identity identifier matching each pedestrian target are determined; further, based on the movement trajectory information corresponding to each identity identifier, passenger flow statistics are performed on each pedestrian target, which solves the problem of being unable to accurately count pedestrian flow when pedestrians hold umbrellas, and realizes accurate pedestrian flow statistics when pedestrians hold umbrellas.

[0039] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0041] Figure 1 This is a hardware structure block diagram of a terminal device for a passenger flow counting method based on umbrella-holding pedestrians provided in an embodiment of the present application;

[0042] Figure 2 is a flow chart of a passenger flow counting method based on umbrella pedestrians provided in one embodiment of the present application;

[0043] Figure 3 is a schematic diagram of a shooting screen of a top-mounted monitoring device provided in an embodiment of the present application;

[0044] Figure 4 is a schematic diagram of a tripwire rule provided by an embodiment of the present application;

[0045] Figure 5 This is a flow chart of a passenger flow counting method based on umbrella-holding pedestrians provided in a preferred embodiment of the present application;

[0046] Figure 6 It is a structural block diagram of a passenger flow counting device based on umbrella-holding pedestrians provided in one embodiment of the present application.

[0047] In the figure: 102, processor; 104, memory; 106, transmission device; 108, input and output device; 10, detection module; 20, tracking module; 30, matching module; 40, statistics module. DETAILED DESCRIPTION

[0048] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0049] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with general skills in the technical field to which this application belongs. The words "one", "a", "the", "these" and the like in this application do not indicate a quantitative limitation, and they may be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" may mean: A exists alone, A and B exist at the same time, and B exists alone. Generally, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0050] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 is a hardware structure block diagram of a terminal of the passenger flow counting method based on umbrella pedestrians in this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 and a memory 104 for storing data, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0051] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the passenger flow counting method based on umbrella pedestrians in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0052] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0053] In this embodiment, a passenger flow counting method based on pedestrians holding umbrellas is provided. Figure 2 is a flow chart of the passenger flow counting method based on umbrella pedestrians in this embodiment. Figure 2 As shown, the process includes the following steps:

[0054] Step S210, detecting each input image by using a well-trained target detection model to obtain a corresponding target label; the target label includes an umbrella label;

[0055] Step S220, tracking pedestrian targets in multiple consecutive frames of input images based on different target labels;

[0056] Step S230, determining a target tag and a corresponding identity identifier that matches each pedestrian target;

[0057] Step S240: performing passenger flow statistics on each pedestrian target based on the movement trajectory information corresponding to each identity identifier.

[0058] Specifically, the entrance and exit of the target location or scene is monitored in real time by monitoring equipment, and images of pedestrians entering and exiting are collected to obtain multiple frames of input images. Each input image is detected by a well-trained target detection model to obtain a corresponding target label. The target label includes at least an umbrella label, and the umbrella label refers to the label obtained by identifying an umbrella when a pedestrian is holding an umbrella. It is usually output in the form of a bounding box such as a rectangular box to facilitate determining the spatial position of the label.

[0059] Among them, the target detection model can adopt the YOLO detection model, the Single Shot MultiBox Detector (SSD) model, etc., which are used to detect the pedestrian head and shoulder labels, the pedestrian holding an umbrella labels, and the umbrella labels in the image. During the training process of the target detection model, the pedestrian head and shoulders, the pedestrian holding an umbrella, and the umbrella in the training image are annotated in the form of bounding boxes such as rectangular boxes, and the model is trained using the annotated training images.

[0060] Furthermore, according to the different target labels output by the model, the pedestrian targets in the continuous multiple frames of input images are tracked. When all the target labels are umbrella labels, since the umbrella used by the pedestrian moves with the movement of the pedestrian, the movement trajectory of the umbrella is relatively synchronized with the movement trajectory of the pedestrian. Based on each umbrella label, the corresponding pedestrian target can be tracked, and the umbrella label matching each pedestrian target is determined. The first identifier of the umbrella label matching the pedestrian target is used as the identity identifier of the pedestrian target, and the first identifier is used to uniquely identify the corresponding umbrella label.

[0061] In addition, when the target labels include pedestrian head and shoulder labels, pedestrian umbrella labels and umbrella labels, since the umbrella used by the umbrella-holding pedestrian moves with the movement of the pedestrian, the position movement trajectories corresponding to the pedestrian head and shoulder labels, the pedestrian umbrella labels and the umbrella labels are relatively synchronized with the pedestrian target. The pedestrian target in the input image can be tracked based on different target labels, and it is determined whether there is an inclusion relationship between the position areas corresponding to each target label. If there is an inclusion relationship between multiple target labels, it indicates that the target labels belong to the same pedestrian target. The target labels are merged and matched to the same pedestrian target, so that multiple target labels matching each pedestrian target can be determined, and the identity of the pedestrian target can be determined based on the target labels corresponding to each pedestrian target.

[0062] Afterwards, the pedestrian target is positioned and tracked based on the identity identifier corresponding to each pedestrian target, and the corresponding movement trajectory information is obtained. According to the movement trajectory information corresponding to each identity identifier and the preset tripwire rules, the entry and exit status of the pedestrian target corresponding to the identity identifier is analyzed, and then based on the entry and exit status of each pedestrian target, passenger flow statistics are performed on each pedestrian target, and the statistical results include the number of pedestrians entering and the number of pedestrians leaving.

[0063] It should be noted that, when the pedestrians in the input image are not holding umbrellas, the pedestrians can be detected and identified, and the detected pedestrian head and shoulder labels can be output, or the detected pedestrian rectangular frame can be output as a pedestrian holding an umbrella label, so that the pedestrians without umbrellas can be tracked subsequently, making this embodiment suitable for the detection of pedestrians with umbrellas and pedestrians without umbrellas.

[0064] Passenger flow counting aims to count and analyze the pedestrian flow at a certain location or place. In the existing passenger flow counting methods, the position of the pedestrian head area is detected and tracked, and the number of people is counted based on the detection and tracking results. However, in the scenario where pedestrians enter and exit with umbrellas, the above method is affected by the umbrellas and cannot accurately count the pedestrian flow when pedestrians are holding umbrellas.

[0065] Compared with the prior art, the present application detects each input image through a trained target detection model to obtain the corresponding target label; the target label includes an umbrella label; based on different target labels, the pedestrian targets in multiple consecutive frames of input images are tracked; the target label and the corresponding identity identifier that match each pedestrian target are determined; further, based on the movement trajectory information corresponding to each identity identifier, passenger flow statistics are performed on each pedestrian target. Based on this, by detecting and obtaining the umbrella label associated with the pedestrian in the input image when the pedestrian is holding an umbrella, the umbrella label is used to accurately track and identify the pedestrian, so that passenger flow statistics can be performed according to the movement trajectory of the pedestrian, which solves the problem of not being able to accurately count pedestrian flow when the pedestrian is holding an umbrella, and realizes accurate pedestrian flow statistics when the pedestrian is holding an umbrella.

[0066] In some embodiments, when each target tag is an umbrella tag, determining the target tag and the corresponding identity matching each pedestrian target in step S230 includes the following steps:

[0067] Determine the umbrella label that matches each pedestrian target;

[0068] The first identifier of the matched umbrella tag is used as the identity identifier of the pedestrian target; the first identifier is used to uniquely identify the corresponding umbrella tag.

[0069] Specifically, when all target tags are umbrella tags, for example, the monitoring device is installed on the top of the entrance, and the input image only contains the picture of the top of the umbrella (such as Figure 3 As shown), determine the umbrella tag that matches each pedestrian target, and use the first identifier of the matched umbrella tag as the identity identifier of the pedestrian target. The first identifier is the identification information of the umbrella tag, which is used to uniquely identify the corresponding umbrella tag.

[0070] It should be noted that when the monitoring equipment is installed on the top of the entrance and exit, due to the fact that the pedestrians do not hold umbrellas or other reasons, the target labels output by the target detection model may also include pedestrian head and shoulder labels and pedestrian umbrella labels. At this time, it is determined whether there is an inclusion relationship between the position areas corresponding to the target labels. If there is an inclusion relationship between the target labels, it indicates that the target labels belong to the same pedestrian target. The target labels are merged and matched to the same pedestrian target, so that multiple target labels matching each pedestrian target can be determined, and the identity of the pedestrian target can be determined based on the target labels corresponding to each pedestrian target.

[0071] Through this embodiment, the umbrella tag that matches each pedestrian target is determined, and the first identifier of the matched umbrella tag is used as the identity identifier of the pedestrian target. Each umbrella tag is used to track and identify the pedestrian target, and when the pedestrian holds an umbrella, the influence of factors such as umbrella obstruction is avoided, the pedestrian target is accurately identified, and the subsequent acquisition of the movement trajectory of each pedestrian target is facilitated.

[0072] In some embodiments, when the target tag also includes a pedestrian head and shoulder tag and a pedestrian holding an umbrella tag, determining the target tag and the corresponding identity identifier that match each pedestrian target in step S230 includes the following steps:

[0073] Determine multiple target labels matching each pedestrian target; wherein there is an inclusion relationship between the location areas corresponding to the target labels;

[0074] Based on the target labels corresponding to each pedestrian target, the identity of the pedestrian target is determined.

[0075] In this embodiment, the target labels output by the target detection model include the label of the head and shoulders of pedestrians, the label of pedestrians holding umbrellas, and the label of umbrellas. For example, the monitoring equipment is installed in an oblique manner to capture the overall motion posture of pedestrians. At this time, the input image collected includes the body parts such as the head, torso, limbs, etc. of the pedestrians and the umbrella holding action of the pedestrians. Accordingly, the target detection result of the input image includes the label of the head and shoulders of pedestrians, the label of pedestrians holding umbrellas, and the label of umbrellas.

[0076] Specifically, determine whether there is an inclusion relationship between the position areas corresponding to each target label. If there is an inclusion relationship between the target labels, it indicates that the target labels belong to the same pedestrian target. The target labels are merged and matched to the same pedestrian target, so that multiple target labels matching each pedestrian target can be determined.

[0077] Furthermore, from among the multiple target tags matching each pedestrian target, a corresponding target tag is selected according to preset priorities of target tags of different categories, and the identifier of the currently selected target tag is used as the identity identifier of the pedestrian target.

[0078] Through this embodiment, multiple target tags matching each pedestrian target are determined, wherein there is an inclusion relationship between the location areas corresponding to the target tags, and the identity of the pedestrian target is determined based on the target tags corresponding to each pedestrian target. In this way, multi-category tags are used to track and identify pedestrian targets. In the case of pedestrians holding umbrellas, the influence of factors such as umbrella obstruction is avoided, and the pedestrian targets are accurately identified, which facilitates the subsequent acquisition of the movement trajectory of each pedestrian target.

[0079] In some embodiments, when the target tag also includes a pedestrian head and shoulder tag and a pedestrian holding an umbrella tag, the step S230 of determining the target tag and the corresponding identity matching each pedestrian target further includes the following steps:

[0080] Determine multiple target tags that match each pedestrian target; wherein there is an inclusion relationship between the position areas corresponding to the target tags, and the position movement trajectories corresponding to the target tags satisfy a preset similarity relationship between each other;

[0081] Based on the target labels corresponding to each pedestrian target, the identity of the pedestrian target is determined.

[0082] In this embodiment, the target labels output by the target detection model include the label of the head and shoulders of pedestrians, the label of pedestrians holding umbrellas, and the label of umbrellas. For example, the monitoring equipment is installed in an oblique manner to capture the overall motion posture of pedestrians. At this time, the input image collected includes the body parts such as the head, torso, limbs, etc. of the pedestrians and the umbrella holding action of the pedestrians. Accordingly, the target detection result of the input image includes the label of the head and shoulders of pedestrians, the label of pedestrians holding umbrellas, and the label of umbrellas.

[0083] Specifically, it is determined whether there is an inclusion relationship between the position areas corresponding to each target tag, and at the same time, the position of each target tag is continuously tracked to obtain the position movement trajectory corresponding to each target tag, and the similarity between the position movement trajectories corresponding to different target tags is calculated to determine whether the position movement trajectories of the two satisfy the preset similarity relationship. If the similarity between the position movement trajectories of the two is greater than a preset threshold, it is determined that the preset similarity relationship is satisfied. The similarity calculation method includes but is not limited to Euclidean distance, edit distance and Hausdorff distance.

[0084] Furthermore, if there is an inclusion relationship between multiple target tags, and the position movement trajectories corresponding to each target tag satisfy the preset similarity relationship, it indicates that the current target tags belong to the same pedestrian target, and the target tags are merged and matched to the same pedestrian target, so that multiple target tags matching each pedestrian target can be determined.

[0085] Afterwards, from the multiple target tags matching each pedestrian target, the corresponding target tag is selected according to the preset priorities of target tags of different categories, and the identifier of the currently selected target tag is used as the identity identifier of the pedestrian target.

[0086] Through this embodiment, multiple target tags matching each pedestrian target are determined, wherein there is an inclusion relationship between the position areas corresponding to the target tags, and the position movement trajectories corresponding to the target tags satisfy a preset similarity relationship pairwise, and based on the target tags corresponding to each pedestrian target, the identity of the pedestrian target is determined, and on the basis of tracking and identifying pedestrian targets using multi-category tags, each target tag is accurately matched to the pedestrian target in combination with the trajectory similarity between different tags, so as to improve the accuracy of identifying pedestrian targets.

[0087] In some embodiments, determining the identity of a pedestrian target based on each target tag corresponding to each pedestrian target includes the following steps:

[0088] Determining multiple target labels matching each pedestrian target;

[0089] Obtain a second identifier of the target tag with the highest priority; the second identifier is used to uniquely identify the corresponding target tag;

[0090] The second identifier is used as the identity identifier of the pedestrian target.

[0091] Specifically, from multiple target tags matching each pedestrian target, according to the preset priorities of target tags of different categories, the target tag with the highest priority is selected, and the second identifier of the target tag with the highest priority is used as the identity identifier of the pedestrian target. The second identifier is the identification information of the target tag, which is used to uniquely identify the corresponding target tag. For example, when the target tag is a pedestrian head and shoulder tag, the second identifier of the target tag is the identification information of the pedestrian head and shoulder tag, which is used to uniquely identify the pedestrian head and shoulder tag. Among them, the priority of each target tag can be preset in a variety of ways, and the priority of the target tag can be set according to the difficulty of detecting different target tags, or based on whether each target tag is convenient for tracking the movement trajectory of the pedestrian target, the priority of the target tag can be set.

[0092] Exemplarily, the priority of the target tag is set based on the difficulty of detection, and the target tags are ranked from high to low in order of priority: pedestrian head and shoulder tag, pedestrian holding umbrella tag, and umbrella tag. Based on this, when the target tags matching the current pedestrian target include pedestrian head and shoulder tag, pedestrian holding umbrella tag, and umbrella tag, the second identifier of the pedestrian head and shoulder tag is output as the identity identifier of the pedestrian target, so as to facilitate the subsequent use of the pedestrian head and shoulder tag to locate and track the movement of the pedestrian target.

[0093] Through this embodiment, multiple target tags matching each pedestrian target are determined, and the second identifier of the target tag with the highest priority is obtained, and the second identifier is used as the identity identifier of the pedestrian target. Therefore, when the pedestrian target matches multiple tags, the preferred tag identifier can be output as the identity identifier of the pedestrian target to improve the accuracy of identifying the pedestrian target.

[0094] In some of the embodiments, determining whether there is an inclusion relationship between the location areas corresponding to the target tags includes the following steps:

[0095] Get the position coordinates of each target tag; the position coordinates include the coordinates of the two diagonal vertices of the rectangular detection box corresponding to the target tag;

[0096] Calculate the position coordinates of target labels of different categories to obtain the inclusion ratio between the position areas corresponding to each target label;

[0097] When the inclusion ratio is greater than a preset threshold, it is determined that there is an inclusion relationship between the target tags.

[0098] In this embodiment, the inclusion ratio between the position areas corresponding to target labels of different categories is calculated, and based on the comparison result of the inclusion ratio with the preset threshold, it is determined whether there is an inclusion relationship between the target labels. The target label is usually output in the form of a bounding box such as a rectangular detection box, and the position area corresponding to the target label is the area within the bounding box.

[0099] Specifically, the position coordinates of each target tag are obtained. Taking a rectangular detection frame as an example, the position coordinates include the coordinates of two diagonal vertices of the rectangular detection frame corresponding to the target tag, such as the coordinates of the upper left corner and the lower right corner of the rectangular detection frame.

[0100] Furthermore, the position coordinates of target tags of different categories are calculated to obtain the inclusion ratio between the position areas corresponding to each target tag. The number of target tags for calculating the inclusion ratio each time is usually two. When the calculated inclusion ratio is greater than the preset threshold, it is determined that there is an inclusion relationship between the target tags. Otherwise, it indicates that there is no inclusion relationship between the target tags.

[0101] For example, the target labels to be judged include the first pedestrian holding an umbrella label and the first umbrella label. The coordinates of the upper left corner of the rectangular box corresponding to the first pedestrian holding an umbrella label are (x 11 ,y 11 ) and the lower right corner coordinates are (x 21 ,y 21 ), the coordinate of the upper left corner of the rectangular box corresponding to the first umbrella label is (x 12 ,y 12 ) and the lower right corner coordinates are (x 22 ,y 22), the inclusion ratio calculation formula between the first pedestrian holding an umbrella label and the first umbrella label is as follows:

[0102]

[0103] In formula (1), Res represents the inclusion ratio between the first pedestrian holding umbrella label and the first umbrella label. When Res is greater than the preset threshold, it is determined that there is an inclusion relationship between the first pedestrian holding umbrella label and the first umbrella label, indicating that the first pedestrian holding umbrella label and the first umbrella label belong to the same pedestrian target, and the first pedestrian holding umbrella label and the first umbrella label can be merged and matched to the same pedestrian target.

[0104] It should be noted that for the same pedestrian target, its pedestrian umbrella label usually includes the pedestrian head and shoulder label and umbrella label related to itself.

[0105] Through this embodiment, the position coordinates of each target tag are obtained, and the position coordinates include the coordinates of the two diagonal vertices of the rectangular detection box corresponding to the target tag. The position coordinates of target tags of different categories are calculated to obtain the inclusion ratio between the position areas corresponding to each target tag. When the inclusion ratio is greater than a preset threshold, it is determined that there is an inclusion relationship between the target tags. The inclusion ratio between the tags is calculated using the position coordinates of each tag, so that it is possible to accurately judge whether there is an inclusion relationship between the tags based on the calculated inclusion ratio.

[0106] In some embodiments, the step S240 of performing passenger flow statistics on each pedestrian target based on the movement trajectory information corresponding to each identity identifier includes the following steps:

[0107] Based on the movement trajectory information corresponding to each identity tag and the preset tripwire rules, determine the entry and exit status of the pedestrian target corresponding to the identity tag;

[0108] Based on the entry and exit status of each pedestrian target, passenger flow statistics are performed on each pedestrian target.

[0109] Specifically, in the continuous multiple frames of input images, the pedestrian targets are located and tracked based on the identity tags corresponding to each pedestrian target, and the movement trajectory information of each pedestrian target is obtained, and the entry and exit status of the pedestrian target is analyzed according to the movement trajectory information and the preset tripwire rule. Among them, the tripwire rule is used to judge the entry and exit status of the pedestrian target according to the relative movement direction generated by the movement trajectory of the pedestrian target when it passes through the preset tripwire at the entrance and exit.

[0110] For example, Figure 4As shown, the preset tripwire divides the entrance and exit area into area 1 and area 2. The tripwire rule stipulates that when the moving trajectory of the pedestrian target is trajectory A, that is, the relative moving direction generated when the pedestrian target crosses the preset tripwire is from area 2 to area 1, it indicates that the pedestrian target is in an entering state and is counted in the total number of pedestrians entering. In addition, the tripwire rule stipulates that when the moving trajectory of the pedestrian target is trajectory B, that is, the relative moving direction generated when the pedestrian target crosses the preset tripwire is from area 1 to area 2, it indicates that the pedestrian target is in a leaving state and is counted in the total number of pedestrians leaving.

[0111] Furthermore, based on the entry and exit status of each pedestrian target, pedestrians in the entry state and pedestrians in the exit state are counted respectively to obtain a final passenger flow statistics result, which includes the number of pedestrians entering and the number of pedestrians leaving.

[0112] It should be noted that in addition to analyzing the entry and exit status of pedestrians based on the tripwire rule, the pedestrian's movement direction can be detected based on the pedestrian's movement trajectory, using one-way sensors, such as infrared sensors, laser sensors, etc., to determine the entry and exit status based on the order in which the pedestrians trigger the sensors.

[0113] Through this embodiment, based on the movement trajectory information corresponding to each identity identification and the preset tripwire rules, the entry and exit status of the pedestrian target corresponding to the identity identification is determined, and based on the entry and exit status of each pedestrian target, passenger flow statistics are performed on each pedestrian target, so that the entry and exit status of pedestrians can be accurately analyzed to achieve accurate statistics of entry and exit passenger flow.

[0114] The present embodiment is described and illustrated below through preferred embodiments.

[0115] Figure 5 : is a flow chart of the passenger flow counting method based on umbrella pedestrians of the preferred embodiment. Figure 5 As shown, the passenger flow counting method based on umbrella pedestrians includes the following steps:

[0116] Step S510, detecting each input image by using a well-trained target detection model to obtain a corresponding target label; the target labels include a pedestrian head and shoulder label, a pedestrian holding an umbrella label, and an umbrella label;

[0117] Step S520, tracking pedestrian targets in multiple consecutive frames of input images based on different target labels;

[0118] Step S530, determining a plurality of target tags matching each pedestrian target; wherein there is an inclusion relationship between the position areas corresponding to the target tags, and the position movement trajectories corresponding to the target tags satisfy a preset similarity relationship between each other;

[0119] Step S540, obtaining a second identifier of the target tag with the highest priority, the second identifier being used to uniquely identify the corresponding target tag; and using the second identifier as the identity identifier of the pedestrian target;

[0120] Step S550, determining the entry and exit status of the pedestrian target corresponding to each identity identifier based on the movement trajectory information corresponding to each identity identifier and the preset tripwire rule;

[0121] Step S560: performing passenger flow statistics on each pedestrian target based on the entry and exit status of each pedestrian target.

[0122] Through this embodiment, each input image is detected by training a complete target detection model to obtain the corresponding target label, which includes a pedestrian head and shoulder label, a pedestrian umbrella label and an umbrella label. Based on different target labels, the pedestrian targets in multiple consecutive frames of input images are tracked to determine multiple target labels that match each pedestrian target; wherein, there is an inclusion relationship between the position areas corresponding to each target label, and the position movement trajectories corresponding to each target label meet the preset similarity relationship, and the second identifier of the target label with the highest priority is obtained, and the second identifier is used as the identity identifier of the pedestrian target, so that on the basis of using multi-category labels to track and identify pedestrian targets, combined with the trajectory similarity between different labels, each target label is accurately matched to the pedestrian target to improve the accuracy of identifying pedestrian targets.

[0123] Furthermore, based on the movement trajectory information corresponding to each identity identification and the preset tripwire rules, the entry and exit status of the pedestrian target corresponding to the identity identification is determined, and based on the entry and exit status of each pedestrian target, passenger flow statistics are performed on each pedestrian target, which solves the problem of being unable to accurately count pedestrian flow when pedestrians hold umbrellas, and realizes accurate pedestrian flow statistics when pedestrians hold umbrellas.

[0124] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0125] In this embodiment, a passenger flow counting device based on umbrella pedestrians is also provided, which is used to implement the above embodiments and preferred implementations, and the descriptions that have been made will not be repeated. The terms "module", "unit", "subunit", etc. used below can be a combination of software and / or hardware that implements the predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0126] Figure 6: is a structural block diagram of the passenger flow counting device based on umbrella pedestrians of this embodiment. Figure 6 As shown, the device includes: a detection module 10, a tracking module 20, a matching module 30 and a statistical module 40;

[0127] The detection module 10 is used to detect each input image by using a well-trained target detection model to obtain a corresponding target label; the target label includes an umbrella label;

[0128] A tracking module 20, configured to track pedestrian targets in a plurality of consecutive frames of input images based on different target labels;

[0129] A matching module 30, for determining a target label and a corresponding identity identifier that matches each pedestrian target;

[0130] The statistics module 40 is used to perform passenger flow statistics on each pedestrian target based on the movement trajectory information corresponding to each identity identifier.

[0131] Through the device provided by this embodiment, each input image is detected by training a complete target detection model to obtain a corresponding target label; the target label includes an umbrella label; based on different target labels, pedestrian targets in multiple consecutive frames of input images are tracked; the target label and the corresponding identity identifier that match each pedestrian target are determined; further, based on the movement trajectory information corresponding to each identity identifier, passenger flow statistics are performed on each pedestrian target, which solves the problem of being unable to accurately count pedestrian flow when pedestrians hold umbrellas, and realizes accurate pedestrian flow statistics when pedestrians hold umbrellas.

[0132] In some of the embodiments, the matching module 30 is further used to determine the umbrella tag that matches each pedestrian target; the first identifier of the matched umbrella tag is used as the identity identifier of the pedestrian target; and the first identifier is used to uniquely identify the corresponding umbrella tag.

[0133] In some of the embodiments, the matching module 30 is further used to determine multiple target tags that match each pedestrian target; wherein there is an inclusion relationship between the location areas corresponding to the target tags; and based on the target tags corresponding to each pedestrian target, the identity of the pedestrian target is determined.

[0134] In some of the embodiments, the matching module 30 is also used to determine multiple target tags that match each pedestrian target; wherein there is an inclusion relationship between the position areas corresponding to the target tags, and the position movement trajectories corresponding to the target tags satisfy a preset similarity relationship between each other; based on the target tags corresponding to each pedestrian target, the identity of the pedestrian target is determined.

[0135] In some of the embodiments, the matching module 30 is also used to determine multiple target tags that match each pedestrian target; obtain the second identifier of the target tag with the highest priority; the second identifier is used to uniquely identify the corresponding target tag; and use the second identifier as the identity identifier of the pedestrian target.

[0136] In some of these embodiments, Figure 6 On the basis of, the device also includes a judgment module, which is used to obtain the position coordinates of each target tag; the position coordinates include the coordinates of two diagonal vertices of the rectangular detection box corresponding to the target tag; the position coordinates of target tags of different categories are calculated to obtain the inclusion ratio between the position areas corresponding to each target tag; when the inclusion ratio is greater than a preset threshold, it is determined that there is an inclusion relationship between the target tags.

[0137] In some of the embodiments, the statistical module 40 is also used to determine the entry and exit status of the pedestrian target corresponding to the identity identifier based on the movement trajectory information corresponding to each identity identifier and the preset tripwire rules; and to perform passenger flow statistics on each pedestrian target based on the entry and exit status of each pedestrian target.

[0138] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0139] In this embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0140] Optionally, the computer device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0141] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0142] S1, detect each input image by training a complete target detection model to obtain the corresponding target label; the target label includes an umbrella label;

[0143] S2, based on different target labels, tracks pedestrian targets in multiple consecutive frames of input images;

[0144] S3, determine the target label and corresponding identity matching each pedestrian target;

[0145] S4, based on the movement trajectory information corresponding to each identity identifier, passenger flow statistics are performed on each pedestrian target.

[0146] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0147] In addition, in combination with the passenger flow counting method based on umbrella-holding pedestrians provided in the above embodiments, a storage medium can also be provided in this embodiment to implement the method. The storage medium stores a computer program; when the computer program is executed by a processor, any of the passenger flow counting methods based on umbrella-holding pedestrians in the above embodiments is implemented.

[0148] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0149] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0150] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0151] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A passenger flow counting method based on pedestrians holding umbrellas, characterized in that: The method comprises: By training a complete target detection model, each input image is detected to obtain the corresponding target label; the target label includes a pedestrian head and shoulder label, a pedestrian holding an umbrella label, and an umbrella label; Tracking pedestrian targets in the input images of multiple consecutive frames based on different target labels; Determining the target label and the corresponding identity identifier that matches each of the pedestrian targets; Wherein, the determining of the target tag and the corresponding identity matching each of the pedestrian targets comprises: determining a plurality of the target tags matching each of the pedestrian targets; wherein there is an inclusion relationship between the position areas corresponding to the target tags, and the position movement trajectories corresponding to the target tags satisfy a preset similarity relationship; and determining the identity of the pedestrian target based on the target tags corresponding to each of the pedestrian targets; Wherein, the determining the identity of the pedestrian target based on each target tag corresponding to each pedestrian target includes: determining a plurality of target tags matching each pedestrian target; setting the priority of each target tag according to the detection difficulty of different target tags; obtaining the second identifier of the target tag with the highest priority; the second identifier is used to uniquely identify the corresponding target tag; and using the second identifier as the identity of the pedestrian target; Based on the movement trajectory information corresponding to each of the identity tags, passenger flow statistics are performed on each of the pedestrian targets.

2. The passenger flow counting method based on umbrella pedestrians according to claim 1 is characterized in that: In the case where each of the target tags is an umbrella tag, determining the target tag and the corresponding identity identifier that matches each of the pedestrian targets includes: Determining the umbrella label that matches each of the pedestrian targets; The first identifier of the matched umbrella tag is used as the identity identifier of the pedestrian target; the first identifier is used to uniquely identify the corresponding umbrella tag.

3. The passenger flow counting method based on umbrella pedestrians according to claim 1 is characterized in that: In the case where the target label further includes a pedestrian head and shoulder label and a pedestrian holding an umbrella label, determining the target label and the corresponding identity identifier that match each pedestrian target includes: Determine a plurality of target tags that match each of the pedestrian targets; wherein there is an inclusion relationship between the position areas corresponding to the target tags; Based on the target tags corresponding to each of the pedestrian targets, an identity of the pedestrian target is determined.

4. The passenger flow counting method based on umbrella pedestrians according to claim 1 or claim 3, characterized in that: Determining whether there is an inclusion relationship between the location areas corresponding to the target tags includes: Obtaining the position coordinates of each target tag; the position coordinates include the coordinates of two diagonal vertices of the rectangular detection box corresponding to the target tag; Calculating the position coordinates of the target tags of different categories to obtain the inclusion ratio between the position areas corresponding to the target tags; When the inclusion ratio is greater than a preset threshold, it is determined that there is an inclusion relationship between the target tags.

5. The passenger flow counting method based on umbrella pedestrians according to claim 1 is characterized in that: The method of performing passenger flow statistics on each of the pedestrian targets based on the movement trajectory information corresponding to each of the identity identifiers includes: Based on the movement trajectory information corresponding to each of the identity identifiers and the preset tripwire rule, determining the entry and exit status of the pedestrian target corresponding to the identity identifier; Based on the entry and exit status of each of the pedestrian targets, passenger flow statistics are performed on each of the pedestrian targets.

6. A passenger flow counting device based on umbrella pedestrians, characterized in that: The device comprises: a detection module, a tracking module, a matching module and a statistical module; The detection module is used to detect each input image by training a complete target detection model to obtain a corresponding target label; the target label includes a pedestrian head and shoulder label, a pedestrian holding an umbrella label, and an umbrella label; The tracking module is used to track pedestrian targets in the input images of multiple consecutive frames based on different target labels; The matching module is used to determine the target label and the corresponding identity identifier that matches each pedestrian target; The matching module is further used to determine a plurality of target tags that match each of the pedestrian targets; wherein there is an inclusion relationship between the position areas corresponding to the target tags, and the position movement trajectories corresponding to the target tags satisfy a preset similarity relationship; based on the target tags corresponding to each of the pedestrian targets, determine the identity of the pedestrian target; The matching module is further used to determine a plurality of target tags that match each of the pedestrian targets; set the priority of each of the target tags according to the detection difficulty of different target tags; obtain the second identifier of the target tag with the highest priority; the second identifier is used to uniquely identify the corresponding target tag; and use the second identifier as the identity identifier of the pedestrian target; The statistical module is used to perform passenger flow statistics on each of the pedestrian targets based on the movement trajectory information corresponding to each of the identity identifiers.

7. A computer device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps of the passenger flow counting method based on umbrella-holding pedestrians according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the passenger flow counting method based on umbrella-holding pedestrians described in any one of claims 1 to 5 are implemented.

Images