Passenger flow human body detection method, device, storage medium and passenger flow counting camera

Abstract

The present application discloses a passenger flow human body detection method, device, storage medium and passenger flow counting camera, which are used to improve the human body detection effect for depth images. The present application method includes: obtaining a pre-output human body detection result, wherein the human body detection result contains a target cluster of a human body area and a bounding box of a human body area; for any two targets detected by the human body detection result of the current frame image, judging whether the following conditions are met: the two targets are adjacent in the vertical direction; and the depth values ​​of the two targets meet the depth characteristics of being close at the top and far at the bottom; if the above conditions are met, determining that the two targets are torn targets; performing a merge operation on the two targets; and re-outputting the human body detection result based on the result of the merge operation.

Description

[0001] This application is a prior application: application number 2022107469999, invention name “passenger flow human body detection method, device, storage medium and passenger flow counting camera”, a divisional application. Technical Field

[0002] The present application relates to the field of image processing technology, and in particular to a passenger flow human body detection method, device, storage medium and passenger flow counting camera. Background Art

[0003] In the fields of consumer electronics, security, transportation, etc., passenger flow statistics are often required to better determine the movement of people. For passenger flow statistics, human body detection is a necessary and crucial link in passenger flow statistics, and its detection accuracy will directly affect the final statistical accuracy.

[0004] In the prior art, human body detection technology based on RGB images is relatively mature. The industry usually adopts detection solutions based on HOG+SVM or deep learning to count passenger flow through RGB images. In the prior art, there is also a method that uses depth cameras based on TOF and structured light to obtain depth images, and then uses depth images for human body detection and passenger flow counting. However, this type of human body detection method based on depth images is simply directly migrated from human body detection methods based on RGB images. Due to the different acquisition principles of depth images and RGB images, there are large differences in imaging, resulting in poor actual detection results, and it is difficult to obtain ideal human body detection results through depth images. Summary of the invention

[0005] In order to solve the above technical problems, the present application provides a passenger flow human body detection method, device, storage medium and passenger flow counting camera.

[0006] The first aspect of the present application provides a passenger flow human body detection method, the method comprising:

[0007] Acquire a pre-output human body detection result, wherein the human body detection result includes a target cluster of a human body region and a bounding box of the human body region;

[0008] Performing data association on the target cluster and the bounding box of the current frame image and the target cluster and the bounding box of the previous frame image to obtain a data association result;

[0009] Determining the adhesion status of individual targets in the human body detection result according to the data association result;

[0010] Based on the adhesion situation, performing a corresponding adhesion splitting operation on the human body detection result;

[0011] Based on the result of the adhesion splitting operation, the human body detection result is re-output.

[0012] Optionally, determining the adhesion status of individual targets in the human body detection result according to the data association result includes:

[0013] When a one-to-many situation exists in the data association result, it is determined that the adhesion situation of the individual targets in the human body detection result is dynamic adhesion.

[0014] Optionally, when there is a one-to-many situation in the data association result, before determining that the adhesion situation of the individual targets in the human body detection result is dynamic adhesion, the method further includes:

[0015] Verify the data association result of the bounding box of the current frame image and the bounding box of the previous frame image;

[0016] If there is a one-to-many situation between the bounding box of the current frame image and the bounding box of the previous frame image, then verify the data association result of the target cluster of the current frame image and the target cluster of the previous frame image;

[0017] If there is a one-to-many situation between the target cluster of the current frame image and the target cluster of the previous frame image, it is determined that the adhesion of the individual targets in the human body detection result is dynamic adhesion.

[0018] Optionally, when it is determined that the adhesion of the individual target in the human body detection result is dynamic adhesion, performing a corresponding adhesion splitting operation on the human body detection result based on the adhesion includes:

[0019] All one-to-many association items in the human body detection result are split, and the following adhesion splitting operation is performed on any of the association items:

[0020] Determine a basic region in the associated item and a region affiliation of the basic region, wherein the basic region is an overlapping region between the target to be split and the associated target;

[0021] Determine a pending area in the associated item, where the pending area is an area of ​​the target to be split excluding the basic area;

[0022] The region affiliation of the undetermined region is determined, thereby obtaining two sub-regions after the split.

[0023] Optionally, associating data between the target cluster and the bounding box of the current frame image and the target cluster and the bounding box of the previous frame image includes:

[0024] Data association is performed through the following formula:

[0025]

[0026] Among them, the Indicates the target in the human detection result

[0027] The overlapping area of ​​A and B, , Represent the areas of targets A and B respectively.

[0028] A second aspect of the present application provides another passenger flow human body detection method, the method comprising:

[0029] Acquire a pre-output human body detection result, wherein the human body detection result includes a target cluster of a human body region and a bounding box of the human body region;

[0030] Determine whether the human body detection result of the current frame image meets the following conditions:

[0031] The ratio between the width and height of the bounding box is within a preset ratio range;

[0032] and

[0033] The target in the bounding box meets the preset head and shoulders features;

[0034] If the above conditions are met at the same time, it is determined that the monomer adhesion condition of the human body detection result is static adhesion;

[0035] Performing an adhesion splitting operation corresponding to the static adhesion on the human body detection result;

[0036] Based on the result of the adhesion splitting operation, the human body detection result is re-output.

[0037] Optionally, determining whether the target in the bounding box satisfies a preset head and shoulder feature includes:

[0038] Taking the upper region of the target as the target region, and calculating the integral projection of the target region;

[0039] Determine the peaks and troughs of the integral projection curve;

[0040] If there is a target trough in the trough, determining that the target in the bounding box satisfies a preset head and shoulder feature;

[0041] The target trough satisfies the following conditions:

[0042] There is a wave crest on both the left and right sides of the target wave trough, and the horizontal spacing and vertical spacing between the target wave trough and the wave crest are both within a preset spacing range.

[0043] Optionally, if there is a trough in the trough, the position of the target trough is stored as a split position of the sticky split operation;

[0044] The performing of the adhesion splitting operation corresponding to the static adhesion on the human body detection result comprises:

[0045] The vertical line at the position of the target trough is used as a dividing line to perform adhesion splitting on the target.

[0046] A third aspect of the present application provides a passenger flow human body detection method, the method comprising:

[0047] Acquire a pre-output human body detection result, wherein the human body detection result includes a target cluster of a human body region and a bounding box of the human body region;

[0048] For any two targets detected by the human body detection results of the current frame image, determine whether the following conditions are met:

[0049] The two targets are adjacent to each other in the vertical direction;

[0050] and

[0051] The depth values ​​of the two targets conform to the depth characteristics of being near at the top and far at the bottom;

[0052] If the above conditions are met, the two targets are determined to be tearing targets;

[0053] performing a merge operation on the two targets;

[0054] Based on the result of the merge operation, the human detection result is re-output.

[0055] Optionally, determining whether the two targets are adjacent to each other in the vertical direction includes:

[0056] Calculating the limit value of the overlapping area of ​​the first target and the second target of the two targets, wherein the limit value includes limits in four directions of upper, lower, left and right, which are upper limit, lower limit, left limit and right limit respectively;

[0057] If the left limit is less than the right limit, then the horizontal overlap ratio and the vertical overlap ratio are calculated respectively;

[0058] If the horizontal overlap ratio and the vertical overlap ratio are both greater than a preset threshold, it is determined that the two targets are adjacent to each other in the vertical direction.

[0059] Optionally, judging whether the depth values ​​of the two targets meet the depth feature of being near at the top and far at the bottom includes:

[0060] Determining the relative upper and lower positions of the first target and the second target of the two targets;

[0061] Based on the depth values ​​of the first target and the second target, determining whether the target above is closer to the camera than the target below;

[0062] If so, it is determined that the depth values ​​of the two targets meet the depth feature of being near at the top and far at the bottom.

[0063] A fourth aspect of the present application provides a passenger counting camera, the passenger counting camera comprising a processor and a depth camera, the processor executing the method as in the first aspect and any optional item in the first aspect during operation.

[0064] A fifth aspect of the present application provides a passenger flow human body detection device, the device comprising:

[0065] A first acquisition unit is used to acquire a pre-output human body detection result, wherein the human body detection result includes a target cluster of a human body region and a bounding box of the human body region;

[0066] A data association unit, used to perform data association on the target cluster and the bounding box of the current frame image with the target cluster and the bounding box of the previous frame image to obtain a data association result;

[0067] an adhesion determination unit, configured to determine the adhesion status of individual targets in the human body detection result according to the data association result;

[0068] A first adhesion splitting unit, configured to perform a corresponding adhesion splitting operation on the human body detection result based on the adhesion situation;

[0069] The first re-output unit is used to re-output the human body detection result based on the result of the adhesion splitting operation.

[0070] A sixth aspect of the present application provides another passenger flow human body detection device, the device comprising:

[0071] A second acquisition unit is used to acquire a pre-output human body detection result, wherein the human body detection result includes a target cluster of a human body region and a bounding box of the human body region;

[0072] The adhesion judgment unit is used to judge whether the human body detection result of the current frame image meets the following conditions:

[0073] The ratio between the width and height of the bounding box is within a preset ratio range;

[0074] and

[0075] The target in the bounding box meets the preset head and shoulders features;

[0076] If the above conditions are met at the same time, it is determined that the monomer adhesion condition of the human body detection result is static adhesion;

[0077] A second adhesion splitting unit, configured to perform an adhesion splitting operation corresponding to the static adhesion on the human body detection result;

[0078] The second re-output unit is used to re-output the human body detection result based on the result of the adhesion splitting operation.

[0079] A seventh aspect of the present application provides another passenger flow human body detection device, the device comprising:

[0080] A third acquisition unit is used to acquire a pre-output human body detection result, wherein the human body detection result includes a target cluster of a human body region and a bounding box of the human body region;

[0081] The tearing judgment unit is used to judge whether any two targets detected by the human body detection result of the current frame image meet the following conditions:

[0082] The two targets are adjacent to each other in the vertical direction;

[0083] and

[0084] The depth values ​​of the two targets conform to the depth characteristics of being near at the top and far at the bottom;

[0085] If the above conditions are met, the two targets are determined to be tearing targets;

[0086] A tearing and merging unit, used for performing a merging operation on the two targets;

[0087] The third re-output unit is used to re-output the human body detection result based on the result of the merging operation.

[0088] In an eighth aspect, the present application provides a passenger flow human body detection device, the device comprising:

[0089] Processor, memory, input-output unit, and bus;

[0090] The processor is connected to the memory, the input and output unit, and the bus;

[0091] The memory stores a program, and the processor calls the program to execute the first aspect and any optional method in the first aspect.

[0092] A ninth aspect of the present application provides a computer-readable storage medium, on which a program is stored. When the program is executed on a computer, the program executes the first aspect and any optional method in the first aspect.

[0093] It can be seen from the above technical solutions that this application has the following advantages:

[0094] The present application provides a method for human body detection in passenger flow. The method can correct the pre-output human body detection results. For example, when correcting, the corresponding adhesion splitting operation can be selected according to the actual adhesion situation. The method has high adaptability and reliability. The method is reliable in detection and can cope with various complex scenes such as missing human body shape and target adhesion and tearing in depth images. In addition, the human body detection method for passenger flow cameras provided by the present application has fast operation speed and low computing power requirements. It can also achieve real-time detection when running on mid- and low-end embedded platform CPUs. It is easy to deploy and has wide applicability, and has the prospect of large-scale promotion and application. BRIEF DESCRIPTION OF THE DRAWINGS

[0095] The following is a description of the drawings involved in this application:

[0096] Figure 1 This is a flow chart of an embodiment of a method for processing dynamic adhesion in this application;

[0097] Figure 2 A schematic diagram of target clusters and bounding boxes in the human body detection results in this application;

[0098] Figure 3 is another schematic diagram of target clusters and bounding boxes in the human body detection results in this application;

[0099] Figure 4 This is a schematic diagram of a sub-flow diagram of an embodiment of step S103 in this application;

[0100] Figure 5 This is a schematic diagram of a sub-flow diagram of an embodiment of step S104 in this application;

[0101] Figure 6 A schematic diagram of a one-to-many association item in this application;

[0102] Figure 7 A schematic diagram of the basic area and the area to be determined in this application;

[0103] Figure 8 This is a schematic diagram of the splitting of the associated items in this application;

[0104] Fig. 9 This is a flow chart of an embodiment of a method for processing static adhesion in the present application;

[0105] Fig.10 This is a schematic diagram of a sub-flow diagram of an embodiment of step S202 in this application;

[0106] Fig.11 A schematic diagram for determining the head and shoulders features in this application;

[0107] Fig.12A schematic diagram of target segmentation in this application;

[0108] Fig.13 This is a schematic flow chart of an embodiment of a method for processing vertical tearing in the present application;

[0109] Fig.14 This is a flow chart of an embodiment of a method for obtaining human body detection results in this application;

[0110] Fig.15 is a schematic diagram of a sub-flow of an embodiment for step S403;

[0111] Fig.16 This is a comparative schematic diagram of using the bounding box of the head instead of the bounding box of the body in this application. DETAILED DESCRIPTION

[0112] Based on this, the present application provides a passenger flow human body detection method for improving the human body detection effect for depth images.

[0113] It should be noted that the human body detection method for passenger flow cameras provided in this application can be applied to passenger flow cameras, depth cameras and other terminals, and can also be applied to servers. Other terminals can be smart phones or computers, tablet computers, smart TVs, smart watches, portable computer terminals, and desktop computers and other smart terminals with computing and data analysis capabilities. For the convenience of explanation, this application uses the terminal as the execution subject for example.

[0114] The human body detection method provided in this embodiment is mainly used in passenger flow cameras, and human body detection and subsequent passenger flow statistics are realized based on depth images. Compared with RGB images, depth images are not easily affected by changes in lighting, and images can be normally collected in dim light or even at night; and depth images do not have information such as color and texture, and will not record a person's appearance information, eliminating people's concerns about privacy; depth images contain distance information, which is convenient for the application of functions based on distance judgment. Therefore, more and more passenger flow cameras use this type of depth camera.

[0115] However, in actual applications, passenger flow cameras need to ensure that they cover a larger detection range, so they are often equipped with depth lenses with large field of view angles. Although the large field of view angle has a wider field of view, the distortion and tilt caused when the human body is close to the edge of the picture will also be more serious; on the other hand, human body parts in depth images, such as the head, body edges, legs and other low-reflection positions are prone to holes or missing, and in severe cases, the shape of the head or legs is completely missing. In other words, the human body in the depth image often does not have robust features, which makes it difficult for conventional machine learning or deep learning solutions for single-frame images to obtain ideal detection results. This embodiment provides a human body detection method for passenger flow cameras that can improve the human body detection effect for depth images.

[0116] See also Figure 1 , Figure 1 A schematic diagram of an embodiment of a passenger flow human body detection method provided in this application, the method includes:

[0117] S101, obtaining a pre-output human body detection result, wherein the human body detection result includes a target cluster of a human body region and a bounding box of the human body region;

[0118] In this embodiment, firstly, a pre-output human body detection result is obtained. The human body detection result is obtained by performing a preliminary detection by a certain method, and includes a target cluster and a bounding box of a human body region. Figure 2 as well as Figure 3 , where the target cluster is a set of pixels in all human body regions, and the bounding box is a rectangular bounding box of the region detected as a human body. Preliminary detection can be performed by the following implementation:

[0119] The input depth image is preprocessed to obtain the image to be detected, the picture motion area in the image to be detected is determined by background modeling, the picture motion area is clustered to determine the human body area set, and the human body detection result is calculated based on the human body area set. Since the result obtained by the preliminary detection may be inaccurate, for example, there may be a situation where the target is adhered. For example, in practice, two people stand in front and behind, which will cause two human body areas to adhere to each other, where adhesion refers to the situation that two or more targets are regarded as one target in the calculated human body detection result because the cluster areas obtained are adhered together due to the close distance of two or more targets; another situation is that the target is torn, and tearing refers to the lack of features, such as the human body area in the depth image is not connected, or the depth value of some human body areas varies greatly and is not continuous (including pedestrians raising their hands, wearing hats, holding umbrellas, etc.), resulting in more than one cluster belonging to the same human body target obtained in the above steps, that is, one target is regarded as two targets. These false detections will eventually affect the accuracy of passenger flow statistics for passenger flow cameras, so it is necessary to correct the human body detection results obtained by the preliminary detection. In this embodiment, correction processing is performed for the situation of target adhesion.

[0120] S102, performing data association on the target cluster and the bounding box of the current frame image and the target cluster and the bounding box of the previous frame image to obtain a data association result;

[0121] In order to determine the adhesion of individual targets in the human detection results, the human detection results of the current frame image (including the target bounding box and target cluster) are data-associated with the human detection results of the previous frame image. If there is no target adhesion in the current scene, the corresponding data association results should all be "one-to-one"; when the target begins to stick, the data association will be "one-to-many". Therefore, by confirming whether there is a "one-to-many" situation in the data association, it can be determined whether the target has gone from separation to adhesion. If there is a one-to-many situation, it means that there is adhesion of individual targets.

[0122] S103, determining the adhesion status of individual targets in the human body detection result according to the data association result;

[0123] The above step S102 mentioned that if there is no target adhesion in the current scene, the corresponding data association results should all be a "one-to-one" relationship; and when the targets begin to stick, the data association will appear to be a "one-to-many" situation. Therefore, the adhesion of individual targets can be determined through the data association results. For example, when a one-to-many situation occurs, the adhesion of individual targets in the human body detection results is determined to be dynamic adhesion. The dynamic adhesion indicates that the targets are separated at the beginning and then adhered together.

[0124] See also Figure 4 In another embodiment, in order to further improve the accuracy of determining the adhesion situation, the method of determining whether the human body detection result has dynamic adhesion can be:

[0125] S1031. When a one-to-many situation exists in the data association result, verify the data association result of the bounding box of the current frame image and the bounding box of the previous frame image. If a one-to-many situation exists between the bounding box of the current frame image and the bounding box of the previous frame image, execute step S1032. If not, execute step 1034 and determine that there is no adhesion.

[0126] S1032, verify the data association result between the target cluster of the current frame image and the target cluster of the previous frame image. If there is a one-to-many situation between the target cluster of the current frame image and the target cluster of the previous frame image, execute step S1033; if there is no one-to-many situation, execute step 1034 and determine that there is no adhesion.

[0127] S1033: Determine that the adhesion status of the individual target in the human body detection result is dynamic adhesion.

[0128] S1034: Determine that there is no adhesion.

[0129] In this embodiment, a dynamic adhesion identification scheme is provided, which adopts a "two-step verification" strategy to effectively improve the identification accuracy.

[0130] In step S1031, the bounding box detected in the current frame image is associated with the bounding box in the previous frame. If a "one-to-many" situation exists, the next step of verification is performed; otherwise, it is directly determined that there is no dynamic adhesion.

[0131] In step S1032, for the targets whose data association is "one to many" in step S1031, the target cluster is used to replace the bounding box for secondary association or confirmation. If the "one to many" situation still exists, it is determined that dynamic adhesion exists, and the data association result is output to the splitting link; otherwise, it is determined that there is no dynamic adhesion.

[0132] Optionally, when performing data association, an IoM (Intersection over Minimum) matching algorithm is used for association matching during the data association process, for example:

[0133] For targets A and B, the IoM calculation formula is:

[0134]

[0135] in, represents the overlapping area of ​​targets A and B, , Respectively represent the areas of targets A and B. Specifically, if data association is performed using bounding boxes, the overlapping area and target area are both calculated based on the bounding boxes; if data association is performed using target clusters, the corresponding overlapping area and target area are both calculated based on the target clusters.

[0136] S104, performing a corresponding adhesion splitting operation on the human body detection result based on the adhesion situation;

[0137] Based on the adhesion situation determined in the above step S103, a corresponding adhesion splitting operation is performed on the human body detection result. For example, when it is determined that the adhesion situation is dynamic adhesion, a splitting operation corresponding to the dynamic adhesion is performed. The splitting operation is used to split the target to be split into several individual targets, thereby obtaining an accurate human body detection result.

[0138] Among them, see Figure 5 In one embodiment, the glue splitting operation may be:

[0139] S1041, split all one-to-many association items in the human body detection result, and perform the following adhesion splitting operation on any of the association items:

[0140] S1042, determining a basic region in the associated item and a region affiliation of the basic region, wherein the basic region is an overlapping region between the target to be split and the associated target;

[0141] S1043, determining a pending region in the associated item, where the pending region is a region of the target to be split excluding the basic region;

[0142] S1044: Determine the region affiliation of the undetermined region, thereby obtaining two sub-regions after the split.

[0143] Based on the data association result obtained in step S102, the "one-to-many" association items are split one by one. The above-mentioned adhesion splitting operation is performed for each "one-to-many" association item, and any association item includes a target to be split and an association target associated with it.

[0144] See also Figure 6 It is a schematic diagram of a "one-to-many" association item. The dotted line in the figure represents the detection result of the previous frame image, and the solid line represents the calculation result of the current frame image. Specifically, , Respectively represent the associated previous frame targets A and B, Indicates the adhesion target detected in the current frame, that is, the target to be split. and , These two targets are related, which belongs to the "one-to-many" situation. Split into two goals.

[0145] In step S1042, the overlapping area between the target to be split and the associated target is calculated and defined as a basic area. Each basic area belongs to the corresponding associated target and is used as the basis for preliminary splitting.

[0146] See also Figure 7 , target to be split Related Target The overlapping area is the basic area A. Related Target The overlapping area of ​​is the basic area B. The basic areas A and B are The preliminary split results.

[0147] In step S1043, the remaining area of ​​the area to be split except the basic area is calculated, which is the area to be determined. Figure 7 middle, The remaining area except the basic areas A and B is the pending area.

[0148] In step S1044, the method of determining the region to be determined may be an outward expansion method or an inward contraction method.

[0149] For example, the area to be determined is shrunk until there is no remaining area. The shrinking process is the process of determining to which target the elements / pixels in the area to be determined belong. For example, the basic area is grown with the boundary of the area to be split as the limit, and the newly added area in each iteration is attributed to the corresponding basic area until there is no remaining area.

[0150] See also Figure 8 , , That is Glue the split target.

[0151] S105: Re-output the human body detection result based on the result of the adhesion splitting operation.

[0152] After performing the corresponding adhesion splitting operation, for example, the target to be split in step S1044 will be split into two sub-areas, and the bounding box and the target cluster are regained. Then, by updating the information of the bounding box and the target cluster, the corrected human detection result can be obtained and the human detection result can be re-output.

[0153] This embodiment provides a method for detecting human bodies in passenger flow. The method is reliable in detection and is combined with a correction mechanism to deal with situations where human body shapes are missing and targets are adhered in depth images, thereby greatly improving the human body detection effect for depth images, thereby ensuring the accuracy of passenger flow statistics.

[0154] In addition, the human body detection method for passenger flow cameras provided in this application has a fast computing speed, low requirements on computing power, and can achieve real-time detection when running on a mid- to low-end embedded platform CPU. It is easy to deploy and has wide applicability, and has the prospect of large-scale promotion and application.

[0155] The method provided by the above step embodiment can handle the problem of dynamic adhesion well, but it is not applicable to static adhesion, that is, the situation where the target is adhered from the beginning. For this reason, this application also proposes an adhesion splitting mechanism for static adhesion, which mainly judges and handles the corresponding adhesion based on the human body features of a single frame. The following will be elaborated in detail:

[0156] See also Fig. 9 , the method provided in this embodiment includes:

[0157] S201, obtaining a pre-output human body detection result, wherein the human body detection result includes a target cluster of a human body region and a bounding box of the human body region;

[0158] In this embodiment, firstly, a pre-output human body detection result is obtained, which is obtained by performing preliminary detection through a certain method, and includes a target cluster and a bounding box of a human body area, wherein the target cluster is a pixel set of all human body areas, and the bounding box is a rectangular bounding box of the area detected as a human body. The preliminary detection can be performed by the following implementation methods:

[0159] The input depth image is preprocessed to obtain the image to be detected, the picture motion area in the image to be detected is determined by background modeling, the picture motion area is clustered to determine the human body area set, and the human body detection result is calculated based on the human body area set. Since the result obtained by the preliminary detection may be inaccurate, for example, there may be a situation where the target is adhered. For example, in practice, two people stand in front and behind, which will cause two human body areas to adhere to each other, where adhesion refers to the situation that two or more targets are regarded as one target in the calculated human body detection result because the cluster areas obtained are adhered together due to the close distance of two or more targets; another situation is that the target is torn, and tearing refers to the lack of features, such as the human body area in the depth image is not connected, or the depth value of some human body areas varies greatly and is not continuous (including pedestrians raising their hands, wearing hats, holding umbrellas, etc.), resulting in more than one cluster belonging to the same human body target obtained in the above steps, that is, one target is regarded as two targets. These false detections will eventually affect the accuracy of passenger flow statistics for passenger flow cameras, so it is necessary to correct the human body detection results obtained by the preliminary detection. In this embodiment, correction processing is performed for the situation of target adhesion.

[0160] The method provided in the above-mentioned embodiment solves the problem of dynamic adhesion well. The method provided in this embodiment is mainly aimed at the situation of static adhesion, that is, the two targets are adhered from the beginning.

[0161] S202: Determine whether the human body detection result of the current frame image meets the following conditions:

[0162] First, the human body detection result of the current frame image is confirmed to determine whether there is static adhesion. The confirmation method is to determine whether the human body detection result meets the following conditions:

[0163] Condition a: the ratio between the width and height of the bounding box is within a preset ratio range;

[0164] Condition b: the target in the bounding box meets the preset head and shoulder features;

[0165] The ratio between the width and height of the bounding box may be an aspect ratio or a height-to-width ratio, and may be constrained by a preset ratio range. For example, the value of the boundary threshold R of the ratio range is usually greater than the conventional single-person aspect ratio.

[0166] Specifically, for any target, first determine whether it meets condition a, and if so, further determine whether it meets condition b.

[0167] For details, see Fig.10 The judgment process of whether condition a is satisfied can be calculated based on the pixel coordinates of the bounding box, and the judgment process of whether condition b is satisfied is:

[0168] S2021, taking the upper area of ​​the target as the target area, and calculating the integral projection of the target area;

[0169] S2022, determining the peaks and troughs of the integral projection curve;

[0170] S2023, if there is a target trough in the trough, determining whether the target in the bounding box satisfies a preset head and shoulder feature;

[0171] The target trough satisfies the following conditions:

[0172] There is a wave crest on both the left and right sides of the target wave trough, and the horizontal spacing and vertical spacing between the target wave trough and the wave crest are both within a preset spacing range;

[0173] If the above conditions are met at the same time, step S204 is executed to determine whether the monomer adhesion of the human body detection result is static adhesion. If there is no target trough, step S206 is executed to determine whether there is no static adhesion.

[0174] See also Fig.11 , take the upper area of ​​the target as the target area, calculate the integral projection of the target area, and optionally, smooth the integral projection result to obtain the optimized integral projection curve; find the peaks and troughs of the integral projection curve; for any trough , if there is a peak on each side (respectively , ),and arrive , If the horizontal and vertical spacing of the wave meet the set spacing range, it is considered to meet the "head and shoulders characteristics"; then the trough The position i is stored as a split position and passed to the splitting stage. Specifically, arrive , The horizontal and vertical spacing setting range is:

[0175]

[0176] in, , are respectively the preset horizontal and vertical spacing thresholds. Preferably, . (For image resolution 160x120)

[0177] S203, determining that the monomer adhesion condition of the human body detection result is static adhesion;

[0178] If the human body detection result satisfies both condition a and condition b, the adhesion situation is determined to be static adhesion, and an adhesion splitting operation corresponding to the static adhesion is performed.

[0179] S204, performing an adhesion splitting operation corresponding to the static adhesion on the human body detection result;

[0180] Split the corresponding sticky targets. For example, if the position i of the trough is determined in the above step, then the position i is stored as the split position. When splitting, the vertical line at the position is used as the dividing line to split the sticky targets. Fig.12 , Fig.12 In the figure, the left side is a schematic diagram of segmentation with a vertical line, and the right side is a schematic diagram of two sub-areas obtained after segmentation. Of course, in practice, segmentation can also be performed with curves based on the characteristics of the target (such as edges), and the present invention does not limit this. The former can be used as a preferred embodiment. Although the shape of the segmentation is relatively rough, the segmentation efficiency is high, and for passenger flow statistics, the main focus is on accurately distinguishing the number of people, and the shape of the human body area does not need to be particularly accurate.

[0181] S205: Re-output the human body detection result based on the result of the adhesion splitting operation.

[0182] After performing the corresponding adhesion splitting operation, the bounding box and the target cluster are regained, and then the information of the bounding box and the target cluster is updated to obtain the corrected human detection result, and the human detection result is re-output.

[0183] The present invention proposes a method for judging and splitting static adhesion. The method mainly judges and processes the corresponding adhesion based on the human body features of a single frame. In practice, human body features refer to the aspect ratio of the bounding box and the "head and shoulder features" side by side. That is, when the aspect ratio of a certain target is too large (exceeding the conventional aspect ratio of a single person), adhesion may exist. Further confirmation is made through the "head and shoulder features" side by side. In this regard, the present invention provides an embodiment for judging static adhesion based on the "head and shoulder features" side by side: the upper area of ​​the target is taken as the target area, and the target area mainly covers the head and shoulder areas of the human body. The integral projection is then calculated for the target area, and the result of the integral projection is used to judge whether the target area has a "peak-trough-peak" that conforms to the "head-shoulder-head" law. If so, it is judged that static adhesion exists, and the "trough" corresponding to the "shoulder" is used as the splitting position to divide the target into two, see. Fig.12 .

[0184] Furthermore, in order to better obtain the peaks and troughs, the integral projection curve may be smoothed, for example, by using median filtering + mean filtering, or by using Kalman filtering, without specific limitation.

[0185] It should be noted that the present application provides methods for judging and splitting dynamic adhesion and static adhesion. In practice, these two methods can be used to correct the human body detection results separately or simultaneously, for example, first judging and splitting dynamic adhesion, and then judging and splitting static adhesion for the human body detection results output after dynamic adhesion splitting. This maximizes the accuracy of the human body detection results.

[0186] The above-mentioned embodiment provides a judgment and splitting method for handling two types of adhesion situations, dynamic adhesion and static adhesion, which can basically cover the vast majority of actual situations and can effectively improve the accuracy of human body detection results. In addition, the method can be applied to the human body detection method of passenger flow cameras. It has low requirements on hardware computing power and does not require hardware modules such as NPU and GPU. It can be directly run on the mid-to-low-end CPU end with fast computing speed. It can also achieve real-time detection when running on the mid-to-low-end embedded platform CPU. It is easy to deploy and has low cost, and has a prospect for large-scale promotion and application.

[0187] However, in some scenarios, such as when the human body area in the depth map is not connected, or the depth values ​​of some areas of the human body vary greatly and discontinuously (including pedestrians raising their hands, wearing hats, holding umbrellas, etc.), the bounding boxes and target clusters of the same person obtained in the above steps may be more than one. This means that there will be false detections for the passenger flow camera, which will ultimately affect the statistical accuracy.

[0188] Therefore, the present invention attributes this problem to "tearing" (corresponding to "adhesion") and proposes a corresponding solution. In practical applications, there may be many types of "tearing", the most common of which is the situation where the same human body is detected as two targets, one above the other. The present invention calls this vertical tearing and provides a corresponding processing mechanism. The following is a detailed description of the embodiment of the method.

[0189] See also Fig.13 , the method provided in this embodiment includes:

[0190] S301, obtaining a pre-output human body detection result, wherein the human body detection result includes a target cluster of a human body region and a bounding box of the human body region;

[0191] In this embodiment, firstly, a pre-output human body detection result is obtained, which is obtained by performing preliminary detection through a certain method, and includes a target cluster and a bounding box of a human body area, wherein the target cluster is a pixel set of all human body areas, and the bounding box is a rectangular bounding box of the area detected as a human body. The preliminary detection can be performed by the following implementation methods:

[0192] The input depth image is preprocessed to obtain the image to be detected, the image motion area in the image to be detected is determined by background modeling, the image motion area is clustered to determine the human body area set, and the human body detection result is calculated based on the human body area set. Since the results obtained by the preliminary detection may be inaccurate, one of the situations is that the target is torn. Tearing refers to the lack of features, such as the human body area in the depth image is not connected, or the depth value of some areas of the human body varies greatly and discontinuously (including pedestrians raising their hands, wearing hats, holding umbrellas, etc.), resulting in more than one cluster belonging to the same human target obtained in the above steps, that is, one target is regarded as two targets. These false detections will eventually affect the accuracy of passenger flow statistics for passenger flow cameras, so it is necessary to correct the human body detection results obtained by the preliminary detection. In this embodiment, correction processing is performed for the situation of target adhesion.

[0193] It should be noted that the principle / basis of the vertical tearing judgment method is:

[0194] In this application, an innovative principle for judging tearing is proposed, that is, for two targets that are adjacent to each other in the vertical direction, if the depth values ​​of the two targets meet the characteristics of "near at the top and far at the bottom", they are usually the same human target, which is a case of vertical tearing. This is mainly because, in the human body detection scene of the passenger flow camera, most of the human bodies are standing, and the distance from the top of the head to the bottom of the feet to the camera generally conforms to the increasing law, that is, "near at the top and far at the bottom", and this law is basically valid in scenes such as raising hands, wearing hats, and holding umbrellas. As a counterexample of vertical tearing, for example, two people standing front and back, although they meet the condition of being adjacent to each other in the vertical direction, due to the front and back standing, the person behind is farther away from the camera than the person in front, so the condition of "near at the top and far at the bottom" is not met, so they can be excluded.

[0195] S302: for any two targets detected by the human body detection result of the current frame image, determine whether the following conditions c and d are met. If so, execute step S303; if not, execute step S306 to determine whether there is no vertical tearing:

[0196] Condition c: the two targets are adjacent to each other in the vertical direction;

[0197] Condition d: the depth values ​​of the two targets meet the depth characteristics of being near at the top and far at the bottom;

[0198] Based on the principle and basis described in step S301, in this embodiment, tearing is judged by using condition c and condition d as constraints. If the detected target satisfies both condition c and condition d, it is considered that the target is torn.

[0199] Specifically, the following provides an embodiment of distinguishing:

[0200] For condition c: calculating the limit value of the overlapping area of ​​the first target and the second target of the two targets, the limit value includes the limits in the four directions of upper, lower, left and right, which are the upper limit, lower limit, left limit and right limit respectively;

[0201] If the left limit is less than the right limit, then the horizontal overlap ratio and the vertical overlap ratio are calculated respectively;

[0202] If the horizontal overlap ratio and the vertical overlap ratio are both greater than a preset threshold, it is determined that the two targets are adjacent to each other in the vertical direction.

[0203] The following example illustrates this:

[0204] For targets A (x1, y1, w1, h1) and B (x2, y2, w2, h2), in this application, the bounding box corresponding to the target is expressed in the form of (x, y, w, h), where x and y represent the horizontal and vertical coordinates of the upper left corner, respectively, and w and h represent the width and height of the bounding box (rectangular box).

[0205] Calculate the left, right, upper and lower limits of the overlapping area:

[0206]

[0207] like Then continue to calculate and judge (otherwise return, determine that it is not vertically adjacent), calculate the horizontal overlap ratio and vertical neighbor ratio :

[0208]

[0209] like and , then it is determined that they are adjacent in the vertical direction. Among them, , are respectively a preset horizontal overlap ratio threshold and a preset vertical proximity ratio threshold. Preferably, .

[0210] If condition c is met, condition d is further determined.

[0211] For condition d: determining the relative upper and lower positions of the first target and the second target of the two targets;

[0212] Based on the depth values ​​of the first target and the second target, determining whether the target above is closer to the camera than the target below;

[0213] If so, it is determined that the depth values ​​of the two targets meet the depth feature of being near at the top and far at the bottom.

[0214] Specifically, the upper and lower positional relationship of the targets (such as A and B) is first determined, and then the depth value is used to determine whether the target above is closer to the camera than the target below. If so, it is determined that the "upper is closer and lower is farther" condition is satisfied.

[0215] S303: Determine that the two targets are tearing targets;

[0216] Further, if the targets meet the above conditions at the same time, step S303 is executed to determine that the two targets are tear targets and associate them. After all targets are determined, all association results are sent to the tear merging stage.

[0217] S304 performs a merge operation on the two targets;

[0218] If there is vertical tearing, the associated target clusters are merged, and the original two target clusters are merged into one. When outputting the human detection results, it will be regarded as a single entity.

[0219] S305 re-outputs the human body detection result based on the result of the merging operation.

[0220] The merged target clusters are recalculated, including calculating the bounding box based on the target clusters and further screening through human feature constraints.

[0221] S306: Determine whether there is no vertical tearing.

[0222] The aforementioned embodiments in the present application are all used to correct the pre-output human body detection results, and the pre-output human body detection results are obtained by preliminary detection through a certain method. The pre-output human body detection results include a target cluster and a bounding box of the human body area, wherein the target cluster is a pixel set of all human body areas, and the bounding box is a rectangular enclosing frame of the area detected as a human body.

[0223] The following provides a specific implementation method for the step of "obtaining a pre-output human body detection result, wherein the human body detection result includes a target cluster of a human body region and a bounding box of the human body region" mentioned in the present application, which is described in detail below. Fig.14 , the embodiment comprises:

[0224] S401, preprocessing the input depth image to obtain an image to be detected;

[0225] The human body detection method provided in this embodiment is mainly used in passenger flow cameras, and human body detection and subsequent passenger flow statistics are realized based on depth images. Compared with RGB images, depth images are not easily affected by changes in lighting, and images can be normally collected in dim light or even at night; and depth images do not have information such as color and texture, and will not record a person's appearance information, eliminating people's concerns about privacy; depth images contain distance information, which is convenient for the application of functions based on distance judgment. Therefore, more and more passenger flow cameras use this type of depth camera.

[0226] However, in actual applications, passenger flow cameras need to ensure that they cover a larger detection range, so they are often equipped with depth lenses with large field of view angles. Although the large field of view angle has a wider field of view, the distortion and tilt caused when the human body is close to the edge of the picture will also be more serious; on the other hand, human body parts in depth images, such as the head, body edges, legs and other low-reflection positions are prone to holes or missing, and in severe cases, the shape of the head or legs is completely missing. In other words, the human body in the depth image often does not have robust features, which makes it difficult for conventional machine learning or deep learning solutions for single-frame images to obtain ideal detection results. This embodiment provides a human body detection method for passenger flow cameras that can improve the human body detection effect for depth images.

[0227] In this embodiment, the terminal first preprocesses the input depth image, and the preprocessing includes but is not limited to downsampling and format conversion of the depth image, so as to reduce the amount of calculation and improve the detection speed.

[0228] The terminal preprocesses the input depth image, and the preprocessing specifically includes downsampling and / or format conversion, and also includes threshold selection on the processed image, and then obtains the image to be detected.

[0229] Specifically, downsampling and / or format conversion includes: if the resolution of the depth image is large, downsampling (i.e., reducing) the original depth image; if the depth image bit depth is greater than 8 bits, converting it to an 8-bit image to reduce the amount of calculation and increase the detection speed. The order of downsampling and format conversion can be interchangeable. Preferably, the terminal first downsamples the original depth image and then converts the downsampled depth image into an 8-bit image.

[0230] Furthermore, the terminal needs to perform depth gating on the depth image after the above processing, that is, pre-set the depth range [Imin, Imax], and set the pixel values ​​that do not meet the depth range to 0. It should be noted that Imin and Imax can be set specifically according to application requirements and actual scenarios. Through depth gating, some dark (close-range) noise and bright (long-range) noise can be preliminarily screened out, which is conducive to improving the detection effect.

[0231] S402, determining the picture motion area in the image to be detected by background modeling;

[0232] The terminal determines the motion area of ​​the image to be detected through background modeling. For scenes where the camera is fixed and the background of the picture changes slowly, background modeling can be directly used for moving target detection or as a pre-processing link to reduce the search range and thus reduce the amount of calculation. The main advantages of background modeling are relatively small amount of calculation and fast speed. In addition, in the depth image obtained in the passenger flow counting scenario, the shape of the human body is often incomplete and not fixed, such as the missing head or legs of the human body. The image of the human body near the edge of the picture obtained by the camera with a large field of view will be distorted and tilted severely. The conventional viewing angle camera based on the single-frame detection method cannot achieve effective human detection. In this embodiment, the motion area of ​​the picture in the image to be detected is determined by background modeling, which can fully consider the information of multiple frames, distinguish the moving target (pedestrian) from the background of the picture, and reduce the interference of the background on the detection results.

[0233] In some specific embodiments, background modeling can be implemented using a codebook algorithm CodeBook or a LOBSTER algorithm.

[0234] S403, clustering the image motion area to determine a human body area set, and calculating a human body detection result based on the human body area set;

[0235] The terminal clusters the image motion region obtained in step S402, specifically clusters the valid pixels in the image motion region, and obtains a set of clusters, namely, the human body region set in this application. Specifically, clustering processing means that for any pixel A already in the cluster, its neighboring pixels are traversed, and for any valid neighboring pixel Ni, if the absolute value of the difference between the pixel value of Ni and A is less than the preset intra-cluster similarity threshold S, Ni is added to the cluster where A is located, otherwise a new cluster is created and Ni is added to it and clustering continues.

[0236] The human body region set obtained above usually contains the main human body regions, but each target cluster cannot be directly regarded as a human body region. This is because:

[0237] 1) The resulting clusters may include non-human areas such as noise areas, moving objects, and misdetected backgrounds;

[0238] 2) When human bodies touch or block each other, the resulting cluster regions are adhered together rather than being individual human body regions, which is referred to as “adhesion” in this application;

[0239] 3) In some scenarios, such as when the human body area in the depth map is not connected or the depth values ​​of some human body areas vary greatly and discontinuously, there may be more than one cluster belonging to the same human target, which is referred to as "tearing" in this application.

[0240] In order to solve the above problem, after obtaining the human body area set, it is necessary to further calculate the corresponding human body detection result based on the human body area set, that is, the terminal traverses the cluster set, obtains the bounding box of all clusters, and uses it as the detection box of the cluster to obtain the human body detection result.

[0241] Specifically, the terminal traverses all pixels of the cluster and obtains the minimum and maximum x and y coordinates of the pixels in the cluster: xmin, ymin, xmax, ymax, and determines the corresponding bounding box rectangle, which is the target detection frame. It should be noted that the human body detection result includes at least a human body region set and a human body bounding box set.

[0242] See also Fig.15 , an embodiment of determining a human body region set is provided below, and the embodiment includes:

[0243] S4031, marking pixels within the picture motion area as valid pixels through an image mask, and marking pixels outside the picture motion area as invalid pixels;

[0244] After obtaining the image motion area, the terminal creates an image mask with the same resolution as the image to be detected to mark whether the pixels are valid. Specifically, the pixels corresponding to the image motion area obtained in step S4032 are marked as valid, and the pixels in the remaining areas are marked as invalid.

[0245] Furthermore, the terminal may also mark the upper, lower, left, and right boundary pixels of the image mask as invalid, so as to avoid boundary checking for each pixel when clustering is subsequently performed, thereby improving efficiency.

[0246] S4032, clustering the effective pixels according to the image mask and the image to be detected to obtain a human body region set;

[0247] The terminal clusters all the pixels with valid marks in step S4031 according to the image mask and the image to be detected to obtain a set of clusters, that is, the human body region set in this application. Specifically, clustering processing means that for any pixel A that is already in the cluster, its neighboring pixels are traversed, and for any valid neighboring pixel Ni, if the absolute value of the difference between the pixel value of Ni and A is less than the preset intra-cluster similarity threshold S, Ni is added to the cluster where A is located, otherwise a new cluster is created and Ni is added to it and clustering continues.

[0248] Furthermore, since clustering involves searching for images, in some specific embodiments, a depth-first search (DFS) or a breadth-first search (BFS) may be used. Preferably, the breadth-first search is used to avoid recursion and consume less memory, so as to improve the calculation speed.

[0249] S4033, determining a bounding box of a human body region set;

[0250] The terminal traverses the human body region set obtained in step S4032, that is, the cluster set, and finds the AABB bounding box of each cluster as the detection frame of the cluster. The process of finding the AABB bounding box is: traverse all pixels of the target cluster, find the minimum and maximum values ​​of the x and y coordinates of the pixels in the cluster: xmin, ymin, xmax, ymax, and then determine the corresponding bounding box rectangle, which is the target detection frame.

[0251] In some specific embodiments, the terminal can further obtain the pixel position corresponding to the highest point of each cluster (whose height is ymax) in the process of obtaining the minimum and maximum values ​​of the x and y coordinates of the pixels in the cluster, and then obtain the coordinate set of the vertices of each cluster. In this way, the highest point can be used as the vertex, and a small head frame can be generated as the top reference to replace the human body bounding box. In this way, in a scene with multiple people and adhesion, it can avoid outputting dense and overlapping detection frames, thereby improving the display effect on the application end.

[0252] See also Fig.16 , Fig.16 The upper part of the diagram is the display before improvement. Fig.16 The lower part of the figure shows the improved display effect after using the head frame instead of the human body bounding box, which greatly improves the user experience.

[0253] S4034, screening the bounding boxes according to the preset constraints, and determining the screened bounding boxes as the human body detection result;

[0254] After calculating the bounding box of the human body region set, the terminal screens the obtained bounding box according to the preset constraints.

[0255] Specifically, the preset constraints include, but are not limited to: human body area constraint, bounding box aspect ratio constraint, boundary limit constraint and height constraint, which will be described below respectively.

[0256] 1) Human body area constraints;

[0257] Specifically, a target area threshold range [Amin, Amax] is set, and targets whose areas are not within the set range are discarded. The target area is the number of pixels in the cluster. Furthermore, the value of Amin should take into account the minimum limit of the area of ​​a single person, while the value of Amax should take into account the maximum limit of the area of ​​a multi-person area in an adhesion scene. This is to take into account the situation of multi-person adhesion, and first retain the area, and leave it for correction in the subsequent adhesion splitting stage. Optionally, the value of Amin can also be smaller than the minimum limit of the area of ​​a single person. The intention is to take into account the situation of tearing of a single human body area, and first retain these results, and leave them for correction in the subsequent tearing merging stage.

[0258] 2) Bounding box aspect ratio constraint;

[0259] Specifically, set the target detection box aspect ratio range [Rmin, Rmax], and discard targets whose detection box aspect ratio is not within the set range. The aspect ratio is the ratio of the width to the height of the bounding box, or the ratio of the height to the width. Furthermore, the value of Rmin should consider the minimum limit of the aspect ratio of the bounding box of a single person, and the value of Rmax should consider the maximum limit of the aspect ratio of the bounding box of multiple people in the adhesion scene, which is the same as the aforementioned [Amin, Amax].

[0260] 3) Boundary limit constraints;

[0261] Specifically, according to application requirements and actual scene characteristics, upper, lower, left, and right boundary lines are set, and targets whose center points exceed the boundary lines are discarded. Specifically, the center point is the center point of the bounding box or the centroid of the human body area, and the center point is preferably the centroid of the human body area. The purpose is to directly ignore the human body when there are many missing shapes at the boundary of the image.

[0262] 4) Height restrictions;

[0263] Specifically, a height threshold HT is set and objects with heights lower than the height threshold are discarded. The purpose is to filter out some low objects that are misdetected, such as chairs that are considered as moving areas by the background modeling module due to movement. There are two specific height discrimination schemes, and the preferred scheme is scheme 2:

[0264] Solution 1: Estimate the actual height of the target in the depth image through the camera's internal and external parameters and the target's pixel coordinates, and then compare the estimated value with the height threshold;

[0265] Solution 2: Use height calibration to collect the depth map of the height plane where the height threshold HT is located to obtain a reference depth map. Alternatively, record the camera height Hc, place the camera at the height of Hc-HT, and then collect the ground depth map as the reference depth map. In actual use, compare the target depth with the reference depth map to determine the height relationship between the target and the reference depth map, so as to determine whether the target is lower than HT.

[0266] It should be noted that in the above screening process, clusters and bounding boxes are always corresponding. When a cluster is screened out, the corresponding bounding box must also be deleted synchronously, and vice versa.

[0267] In practical applications, step S4034 may be executed multiple times, but the height constraint does not need to be executed every time, and it only needs to be executed once in the total detection process. Specifically, after executing the corresponding splitting or merging process, it is necessary to recalculate the bounding boxes of all clusters and filter the results by the human body area constraint, bounding box aspect ratio and boundary limit constraint, while filtering the detection results by the height constraint is placed after the above steps as a separate step and is only executed once.

[0268] The present application also provides a passenger flow camera, which includes a processor and a depth camera. The processor executes any of the above passenger flow human body detection methods during operation.

[0269] The present application also provides a passenger flow human body detection device, the device comprising:

[0270] A first acquisition unit is used to acquire a pre-output human body detection result, wherein the human body detection result includes a target cluster of a human body region and a bounding box of the human body region;

[0271] A data association unit, used to perform data association on the target cluster and the bounding box of the current frame image with the target cluster and the bounding box of the previous frame image to obtain a data association result;

[0272] an adhesion determination unit, configured to determine the adhesion status of individual targets in the human body detection result according to the data association result;

[0273] A first adhesion splitting unit, configured to perform a corresponding adhesion splitting operation on the human body detection result based on the adhesion situation;

[0274] The first re-output unit is used to re-output the human body detection result based on the result of the adhesion splitting operation.

[0275] The present application also provides another passenger flow human body detection device, the device comprising mutually coupled:

[0276] A second acquisition unit is used to acquire a pre-output human body detection result, wherein the human body detection result includes a target cluster of a human body region and a bounding box of the human body region;

[0277] The adhesion judgment unit is used to judge whether the human body detection result of the current frame image meets the following conditions:

[0278] The ratio between the width and height of the bounding box is within a preset ratio range;

[0279] and

[0280] The target in the bounding box meets the preset head and shoulders features;

[0281] If the above conditions are met at the same time, it is determined that the monomer adhesion condition of the human body detection result is static adhesion;

[0282] A second adhesion splitting unit, configured to perform an adhesion splitting operation corresponding to the static adhesion on the human body detection result;

[0283] The second re-output unit is used to re-output the human body detection result based on the result of the adhesion splitting operation.

[0284] The present application also provides another passenger flow human body detection device, the device comprising mutually coupled:

[0285] A third acquisition unit is used to acquire a pre-output human body detection result, wherein the human body detection result includes a target cluster of a human body region and a bounding box of the human body region;

[0286] The tearing judgment unit is used to judge whether any two targets detected by the human body detection result of the current frame image meet the following conditions:

[0287] The two targets are adjacent to each other in the vertical direction;

[0288] and

[0289] The depth values ​​of the two targets conform to the depth characteristics of being near at the top and far at the bottom;

[0290] If the above conditions are met, the two targets are determined to be tearing targets;

[0291] A tearing and merging unit, used for performing a merging operation on the two targets;

[0292] The third re-output unit is used to re-output the human body detection result based on the result of the merging operation.

[0293] The present application also provides a passenger flow human body detection device, comprising:

[0294] Processor, memory, input and output unit, bus;

[0295] The processor is connected to the memory, the input and output unit, and the bus;

[0296] The memory stores a program, and the processor calls the program to execute any of the above passenger flow human body detection methods.

[0297] The present application also relates to a computer-readable storage medium, on which a program is stored, characterized in that when the program is run on a computer, the computer is caused to execute any of the above methods. The device embodiments described above are merely illustrative, and the division of the units is merely a logical function division, and there may be other division methods in actual implementation.

Claims

1. A passenger flow human body detection method, characterized in that: The method comprises: Acquire a pre-output human body detection result, wherein the human body detection result includes a target cluster of a human body region and a bounding box of the human body region; For any two targets detected by the human body detection results of the current frame image, determine whether the following conditions are met: The two targets are adjacent to each other in the vertical direction; and The depth values ​​of the two targets conform to the depth characteristics of being near at the top and far at the bottom; If the above conditions are met, the two targets are determined to be tearing targets; performing a merge operation on the two targets; Based on the result of the merging operation, re-outputting the human body detection result; judging whether the two targets are adjacent in the vertical direction includes: Calculating the limit value of the overlapping area of ​​the first target and the second target of the two targets, wherein the limit value includes limits in four directions of upper, lower, left and right, which are upper limit, lower limit, left limit and right limit respectively; If the left limit is less than the right limit, then the horizontal overlap ratio and the vertical overlap ratio are calculated respectively; If the horizontal overlap ratio and the vertical overlap ratio are both greater than a preset threshold, it is determined that the two targets are adjacent in the vertical direction; Determining whether the depth values ​​of the two targets meet the depth characteristics of being near at the top and far at the bottom includes: Determining the relative upper and lower positions of the first target and the second target of the two targets; Based on the depth values ​​of the first target and the second target, determining whether the target above is closer to the camera than the target below; If so, it is determined that the depth values ​​of the two targets meet the depth feature of being near at the top and far at the bottom.

2. A passenger counting camera, characterized in that: The passenger flow counting camera includes a processor and a depth camera, and the processor executes the passenger flow human body detection method as claimed in claim 1 during operation.

3. A passenger flow human body detection device, characterized in that: The device comprises: A third acquisition unit is used to acquire a pre-output human body detection result, wherein the human body detection result includes a target cluster of a human body region and a bounding box of the human body region; The tearing judgment unit is used to judge whether any two targets detected by the human body detection result of the current frame image meet the following conditions: The two targets are adjacent to each other in the vertical direction; and The depth values ​​of the two targets conform to the depth characteristics of being near at the top and far at the bottom; If the above conditions are met, the two targets are determined to be tearing targets; A tearing and merging unit, used for performing a merging operation on the two targets; A third re-output unit, used to re-output the human body detection result based on the result of the merging operation; The tear judgment unit is specifically used for: Calculating the limit value of the overlapping area of ​​the first target and the second target of the two targets, wherein the limit value includes limits in four directions of upper, lower, left and right, which are upper limit, lower limit, left limit and right limit respectively; If the left limit is less than the right limit, then the horizontal overlap ratio and the vertical overlap ratio are calculated respectively; If the horizontal overlap ratio and the vertical overlap ratio are both greater than a preset threshold, it is determined that the two targets are adjacent in the vertical direction; The tear judgment unit is specifically used for: Determining the relative upper and lower positions of the first target and the second target of the two targets; Based on the depth values ​​of the first target and the second target, determining whether the target above is closer to the camera than the target below; If so, it is determined that the depth values ​​of the two targets meet the depth feature of being near at the top and far at the bottom.

4. A passenger flow human body detection device, characterized in that: The device comprises: Processor, memory, input-output unit, and bus; The processor is connected to the memory, the input and output unit, and the bus; The memory stores a program, and the processor calls the program to execute the passenger flow human body detection method as claimed in claim 1.

5. A computer-readable storage medium having a program stored thereon, wherein the program, when executed on a computer, performs the passenger flow human body detection method as claimed in claim 1.

Images