Human body distribution detection method, aerial photography equipment, electronic device and storage medium

Abstract

The present application relates to a human body distribution detection method, aerial photography equipment, electronic device and storage medium. A first video stream collected by a device moving in the air at a bird's-eye view angle is obtained, and human body detection is performed on the first video stream to obtain human body position information of each human body among multiple human bodies; each human body is classified according to the human body position information, and the human body state of each human body is determined according to the classification result, wherein the human body state includes a non-isolated state and an isolated state; route information of the device moving in the air is obtained, and a target area in each video frame in the first video stream is determined according to the route information, and human body distribution characteristics in the target area are determined according to the human body position information and the human body state of each human body, thereby solving the problem of low accuracy of human body distribution characteristics obtained based on bird's-eye view angle detection and improving the accuracy of detecting human body distribution characteristics.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a human body distribution detection method, aerial photography equipment, electronic device and storage medium. Background Art

[0002] Dense crowd detection uses an aerial robot equipped with vision to detect and count small targets in the heads of densely packed people.

[0003] Related technologies provide a method for detecting dense crowds based on drones, which uses drones equipped with embedded GPU (graphics processing unit) computing modules to replace fixed servers and fixed monitoring equipment; and uses deep learning algorithms to calculate crowd density distribution maps to calculate the number of people. Its disadvantages are: using crowd density distribution maps to calculate the number of people requires that the image shooting height and angle are relatively stable, which can be achieved when the drone is hovering and equipped with a high-quality gimbal, but this method sacrifices the flexible movement of the drone, and the density distribution map method cannot specifically count individual information, which easily causes the calculation results to differ greatly from the actual situation.

[0004] Currently, no effective solution has been proposed to address the problem of low accuracy of pedestrian distribution features detected based on a bird's-eye view in related technologies. Summary of the invention

[0005] In this embodiment, a human body distribution detection method, an aerial photography device, an electronic device, and a storage medium are provided to solve the problem of low accuracy of human body distribution features obtained based on bird's-eye view detection in the related art.

[0006] In a first aspect, a method for detecting human body distribution is provided in this embodiment, comprising:

[0007] Acquire a first video stream captured by a device moving in the air at a bird's-eye view angle, perform human body detection on the first video stream, and obtain human body position information of each human body among a plurality of human bodies;

[0008] Classifying each human body according to the human body position information of each human body, and determining the human body state of each human body according to the classification result, wherein the human body state includes a non-isolated state and an isolated state;

[0009] The route information of the device moving in the air is obtained, the target area in each video frame in the first video stream is determined according to the route information, and the human body distribution characteristics in the target area are determined according to the human body position information and the human body state of each human body.

[0010] In some embodiments, the route information includes a corresponding timestamp and a route position, and determining the target area in each video frame in the first video stream according to the route information includes:

[0011] Divide the first video stream according to the timestamp to obtain individual video frames;

[0012] The geographical location corresponding to the target area is determined according to the route position, and the geographical location is mapped to each video frame to obtain the target area in each video frame.

[0013] In some embodiments, the target area includes a safe area and a non-safe area. After determining the target area in each video frame in the first video stream according to the route information, and determining the human body distribution characteristics in the target area according to the human body position information and the human body state of each human body, the method further includes:

[0014] Determine that a human body whose state is the non-isolated state and whose position information belongs to the safe area is a safe object; and

[0015] A human body whose state is the isolated state and whose position information belongs to the unsafe area is determined to be a dangerous object.

[0016] In some embodiments, after determining that a human body whose state is the non-isolated state and whose position information belongs to the safe area is a safe object; and determining that a human body whose state is the isolated state and whose position information belongs to the non-safe area is a dangerous object, the method further includes:

[0017] Determining the number of the safe objects and obtaining the location information of the dangerous objects;

[0018] Report the number of the safe objects and the location information of the dangerous objects.

[0019] In some embodiments, performing human body detection on the first video stream to obtain human body position information of each of the multiple human bodies includes:

[0020] The first video stream is input into a trained deep convolutional neural network, target detection is performed on the first video stream with the human head as the target based on the deep convolutional neural network, each of the multiple heads detected is determined as the respective human body, and the position information of the each detected head is determined as the human position information of the respective human body.

[0021] In some embodiments, inputting the first video stream into a trained deep convolutional neural network, and performing target detection on the first video stream with the human head as the target based on the deep convolutional neural network includes:

[0022] Introducing a channel attention mechanism into the deep convolutional neural network, and determining the weight of each feature channel via the channel attention mechanism;

[0023] According to the weight of each feature channel, the first video stream is downsampled multiple times to obtain a multi-scale feature map with channel attention;

[0024] Feature fusion is performed on the multi-scale feature map to output feature information of multiple human heads.

[0025] In some embodiments, before inputting the first video stream into the trained deep convolutional neural network and performing target detection on the first video stream with the human head as the target based on the deep convolutional neural network, the method further includes:

[0026] Acquire a second video stream captured by a device moving in the air at a bird's-eye view;

[0027] Annotating the head of the human body in the second video stream, and dividing the annotated video stream into a training set and a test set according to a preset ratio;

[0028] Constructing a deep convolutional neural network, wherein the deep convolutional neural network includes N downsampling units, wherein the first M downsampling units are provided with a channel attention unit, and N>M;

[0029] The deep convolutional neural network is trained according to the training set and the test set to obtain a trained deep convolutional neural network.

[0030] In some embodiments, classifying each human body according to the human body position information of each human body, and determining the human body state of each human body according to the classification result includes:

[0031] Acquire height information of the device relative to the ground, and determine a neighborhood radius according to the height information and a preset minimum number of included points;

[0032] Acquire a first feature point cluster with a human head as a feature point, select a first feature point that has not been visited in the first feature point cluster, determine adjacent feature points within a neighborhood range of the first feature point according to the neighborhood radius and positions of other feature points except the first feature point, and generate a second feature point cluster according to the first feature point and adjacent feature points within the neighborhood range of the first feature point;

[0033] Determining that feature points in the first feature point cluster and the second feature point cluster belong to a first category, and determining that feature points outside the first feature point cluster and the second feature point cluster belong to a second category;

[0034] The human body state corresponding to the feature point of the first category is determined to be the non-isolated state, and the human body state corresponding to the feature point of the second category is determined to be the isolated state.

[0035] In some embodiments, obtaining a first feature point cluster, selecting a first feature point that has not been visited in the first feature point cluster, determining adjacent feature points within a neighborhood range of the first feature point according to the neighborhood radius and positions of other feature points except the first feature point, and generating a second feature point cluster according to the first feature point and adjacent feature points within the neighborhood range of the first feature point includes:

[0036] Determine whether the number of adjacent feature points within the neighborhood of the first feature point reaches the preset minimum number of included points;

[0037] When it is determined that the number of adjacent feature points within the neighborhood of the first feature point reaches the preset minimum number of included points, the second feature point cluster is generated based on the first feature point and the adjacent feature points within the neighborhood of the first feature point, and the first feature point that has been visited is marked.

[0038] In some embodiments, when it is determined that the number of adjacent feature points within the neighborhood of the first feature point does not reach the preset minimum number of included points, the method further includes:

[0039] The first feature point is determined to be a noise point, and the first feature point is determined to belong to the second category.

[0040] In some embodiments, obtaining a first feature point cluster using a human head as a feature point includes:

[0041] Selecting a first feature point from the plurality of feature points for which no feature point has been accessed;

[0042] Determine adjacent feature points within the neighborhood range of the first feature point according to the neighborhood radius and positions of other feature points except the first feature point;

[0043] The first feature point cluster is generated according to the first feature point and adjacent feature points within a neighborhood range of the first feature point.

[0044] In the second aspect, an aerial photography device is provided in this embodiment, including a flying body, the flying body equipped with a camera and a processor connected to each other; wherein the camera is used to shoot a video stream; and the processor is used to execute the human body distribution detection method described in the first aspect above.

[0045] In a third aspect, an electronic device is provided in this embodiment, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the human body distribution detection method described in the first aspect when executing the computer program.

[0046] In a fourth aspect, in this embodiment, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the steps of the human body distribution detection method described in the first aspect are implemented.

[0047] Compared with the related art, the human body distribution detection method, aerial photography equipment, electronic device and storage medium provided in the present embodiment obtain a first video stream collected by a device moving in the air at a bird's-eye view angle, perform human body detection on the first video stream, and obtain human body position information of each human body among multiple human bodies; classify each human body according to the human body position information of each human body, and determine the human body state of each human body according to the classification result, wherein the human body state includes a non-isolated state and an isolated state; obtain the route information of the device moving in the air, determine the target area in each video frame in the first video stream according to the route information, and determine the human body distribution characteristics in the target area according to the human body position information of each human body and the human body state of each human body, thereby solving the problem of low accuracy of human body distribution characteristics obtained based on bird's-eye view angle detection and improving the accuracy of detecting human body distribution characteristics.

[0048] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0050] Figure 1 is a hardware structure block diagram of a terminal of the human body distribution detection method of this embodiment;

[0051] Figure 2 is a flow chart of the human body distribution detection method of this embodiment;

[0052] Figure 3It is a training flow chart of the YOLOv3 deep convolutional neural network of this embodiment;

[0053] Figure 4 Schematic diagram of the structure of the YOLOv3 deep convolutional neural network of this embodiment;

[0054] Figure 5 is an adaptive density clustering method based on device height of an embodiment;

[0055] Figure 6 is a schematic diagram of the adaptive density clustering result based on device height in this embodiment;

[0056] Figure 7 4 is a flow chart of the method for detecting human body distribution based on drone of the preferred embodiment. DETAILED DESCRIPTION

[0057] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0058] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0059] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1FIG. 1 is a hardware structure diagram of a terminal of the human body distribution detection method of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 and memory 104 for storing data, wherein processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0060] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the human body distribution detection method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0061] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0062] In this embodiment, a method for detecting human body distribution is provided. Figure 2 is a flow chart of the human body distribution detection method of this embodiment, such as Figure 2 As shown, the process includes the following steps:

[0063] Step S1, obtaining a first video stream captured by a device moving in the air at a bird's-eye view, performing human body detection on the first video stream, and obtaining human body position information of each human body among a plurality of human bodies.

[0064] The device moving in the air may be an aerial photography device such as a drone or a radar. When the aerial photography device flies in the air, it photographs the land and / or the river or the sea from a bird's-eye view to obtain a first video stream.

[0065] When detecting the human body in the first video stream, the human body can be detected by manually designing features or by deep network feature learning, which is not limited in this embodiment. The human body position information is the position relative to each video frame in the video stream, rather than the actual geographical location.

[0066] Step S2, classifying each human body according to the human body position information of each human body, and determining the human body state of each human body according to the classification result, wherein the human body state includes a non-isolated state and an isolated state.

[0067] In this embodiment, when the distance between multiple human bodies in a group is less than a first threshold and / or the number of the multiple human bodies is greater than a second threshold, the human body state of each human body in the group is defined as a non-isolated state, and the human body state of each human body not in the group is defined as an isolated state.

[0068] In specific implementation, a certain part of the human body can be used as a feature point, and a density-based spatial clustering algorithm (DBSCAN) can be used to cluster the feature points, and the feature points that can form a cluster with other feature points are classified as non-isolated feature points, and the feature points that cannot form a cluster with other feature points are classified as isolated feature points. Among them, the feature points that can form a cluster with each other meet the preset conditions, for example, the distance between these feature points is less than the first threshold, and / or the number of feature points belonging to the same cluster is greater than the second threshold.

[0069] Step S3, obtaining the route information of the device moving in the air, determining the target area in each video frame in the first video stream according to the route information, and determining the human body distribution characteristics in the target area according to the human body position information and human body state of each human body.

[0070] The route information is information used to guide equipment inspections. The route information includes a corresponding timestamp and route position, that is, the flight time and flight path of the equipment are planned. In some embodiments, the first video stream can be divided according to the timestamp to obtain each video frame, and the geographical location corresponding to the target area can be determined according to the route position, and the geographical location can be mapped to each video frame, that is, the geographical location of the target area is converted into a position relative to the video frame to obtain the target area in each video frame. When the geographical location of the target area is known, the target area of ​​each video frame can be updated in real time through the route information. By judging the positional relationship between the human body and the target area, it can be known whether the human body is in the target area. Based on the current target area and the state of the human body, the distribution characteristics of the human body in the target area can be obtained.

[0071] The human body distribution detection method of this embodiment can be applied to monitoring scenarios of some traffic sections or scenic spots to provide technical support for the governance work of relevant management departments.

[0072] Compared with the related art that uses density distribution graphs to detect crowds, this embodiment detects the state of the human body and distinguishes between the human body in a non-isolated state and the human body in an isolated state, so that individual information can be counted and the error between the calculation result and the actual situation can be reduced. In addition, the target area in the video frame is updated according to the route information, and the distribution characteristics of the human body in the target area are obtained based on the current target area and the state of the human body. Through the detection of these two dimensions of information, the real-time distribution of the human body in the target area is obtained, which improves the real-time nature of the human body distribution characteristics. Through the above steps, the problem of low accuracy of the human body distribution characteristics obtained based on the overhead angle detection is solved, and the accuracy of detecting the human body distribution characteristics is improved.

[0073] In some embodiments, the target area can be divided into multiple areas according to actual needs, such as a safe area and an unsafe area, and the human body distribution characteristics in the target area can be divided into the following situations: there are non-isolated human bodies in the safe area; there are isolated human bodies in the safe area; there are non-isolated human bodies in the unsafe area; there are isolated human bodies in the unsafe area.

[0074] After determining the human body distribution characteristics in the target area, a human body whose state is non-isolated and whose position information belongs to a safe area is determined to be a safe object; and a human body whose state is isolated and whose position information belongs to a non-safe area is determined to be a dangerous object.

[0075] Furthermore, after obtaining the above-mentioned human body distribution characteristics, by determining the number of safe objects and obtaining the position information of dangerous objects, the number of safe objects and the position information of dangerous objects are reported.

[0076] When reporting the number of safe objects and the location information of dangerous objects, the current location can be obtained through the global positioning system (GPS) of the device, and the current location can be reported to the user as the location information of the dangerous object. The user can be a server, such as a ground command center, or a portable device, such as a smart phone. The on-duty personnel can obtain the human distribution characteristics of the target area through the ground command center or the smart phone, and then the on-duty personnel will drive the dangerous objects away from the unsafe area. In this way, the human-machine collaboration capability is improved through the combination of "aerial equipment inspection + ground personnel expulsion", which helps to improve the pedestrian monitoring and management capabilities of traffic sections and scenic spots.

[0077] In some preferred embodiments, a multi-rotor drone can be used to fly a preset route, and the multi-rotor drone can inspect and shoot densely populated road sections. According to the division characteristics of such road sections, the movable area of ​​the human body is divided into a safe area, and the non-movable area of ​​the human body is divided into an unsafe area. For the video stream shot by the multi-rotor drone, the coordinate information of each boundary and key area in each frame of the video image in the video stream is determined according to the route information, and the division range of the safe area and the unsafe area in the video image is continuously updated to obtain real-time human distribution characteristics.

[0078] In step S1, the first video stream can be input into the trained deep convolutional neural network, and target detection with the human head as the target is performed on the first video stream based on the deep convolutional neural network, and each of the multiple heads detected is determined as a human body, and the position information of each detected head is determined as the human body position information of each human body. Compared with the related art using the manually designed feature extraction method, this embodiment uses a deep convolutional neural network to extract human head features, avoiding the complex and inefficient problems of manually designed features and improving scene adaptability.

[0079] Furthermore, in view of the fact that the heads of densely populated people are small targets from the perspective of drones, which may easily cause target jitter and occupy fewer pixels, thus leading to a problem of too low target recall rate, in step S1, a channel attention mechanism can be introduced into the deep convolutional neural network, and the weights of each feature channel can be determined through the channel attention mechanism; according to the weights of each feature channel, the first video stream is downsampled multiple times to obtain a multi-scale feature map with channel attention; the multi-scale feature map is feature fused to output feature information of multiple human heads.

[0080] Among them, the training process of the deep convolutional neural network is as follows: obtain a second video stream collected by a device moving in the air at a bird's-eye view; mark the head of the human body in the second video stream, and divide the marked video stream into a training set and a test set according to a preset ratio; construct a deep convolutional neural network, the deep convolutional neural network includes N downsampling units, among which the first M downsampling units are provided with channel attention units, N>M; train the deep convolutional neural network according to the training set and the test set to obtain the trained deep convolutional neural network.

[0081] The following will introduce the training process of a deep convolutional neural network through a preferred embodiment.

[0082] In some preferred embodiments, YOLOv3 (You Only Look Once v3, the third version of the target detection algorithm) can be used as the basic structure to build a deep convolutional neural network, and the feature information of the feature points can be extracted through the YOLOv3 deep convolutional neural network. Figure 3 is a training flow chart of the YOLOv3 deep convolutional neural network of this embodiment, such as Figure 3 As shown, the process includes the following steps:

[0083] Step S11, collect 5000 video frames taken from the perspective of the drone, annotate the human head, and randomly divide the annotated video frames into a training set and a test set in a ratio of 8:2. Each video frame contains human bodies of different numbers, postures, and costumes.

[0084] Step S12, constructing a YOLOv3 deep convolutional neural network. Figure 4 : is a schematic diagram of the structure of the YOLOv3 deep convolutional neural network of this embodiment, such as Figure 4 As shown, a channel attention module is provided in the YOLOv3 deep convolutional neural network, and the channel attention module includes: Darknetconv2D_2BN_Leaky unit, SE_Res_unit unit and SE_Resblock_body unit.

[0085] The Darknetconv2D_2BN_Leaky unit includes: convolutional layer, batch normalization layer and leaky relu activation function.

[0086] The SE_Res_unit unit includes: a basic attention-residual unit consisting of 2 DBLs, 1 channel attention unit (SE), and a jump structure.

[0087] The SE_Resblock_body unit includes: padding layer, convolutional layer and n SE_Res_unit units.

[0088] Among them, DBL stands for the abbreviation of Darknetconv2D_2BN_Leaky.

[0089] se_res1 represents a SE_Resblock_body (attention-residual block) containing 1 SE_Res_unit unit, se_res2 represents a SE_Resblock_body (attention-residual block) containing 2 SE_Res_unit units, and se_res8 represents a SE_Resblock_body (attention-residual block) containing 8 SE_Res_unit units.

[0090] res4 represents a Resblock_body (residual block) containing 4 Res_unit units, and res8 represents a Resblock_body (residual block) containing 8 Res_unit units.

[0091] y1, y2, and y3 represent the feature layers of three scales output by the network.

[0092] zero_padding represents the zero padding layer.

[0093] se_resn represents a residual block containing n SE_Res_units.

[0094] BN stands for BatchNormalization.

[0095] leaky_relu represents an activation function.

[0096] concat represents the feature superposition function.

[0097] add represents a jump function.

[0098] SE stands for attention unit.

[0099] Introduce channel attention units at specific positions of the first three downsampling units of the Darknet-53 (YOLOv3 feature extractor, containing 53 convolutional layers) feature extraction network, for example, SENet (Squeeze-and-Excitation Network) is used as the channel attention unit, where Squeeze represents an operation that compresses features along the spatial dimension, converting each two-dimensional feature channel into a real number that has a global receptive field to some extent, and the output dimension matches the number of input feature channels. Excitation represents an operation that is similar to a gate mechanism in a recurrent neural network, generating weights for each feature channel through a parameter w, where the parameter w is learned to explicitly model the correlation between feature channels.

[0100] In step S13, after the video frame is downsampled five times by the Darknet-53 feature extraction network, three scale feature maps are output, and feature maps that can be directly predicted are output after feature fusion by FPN (Feature Pyramid Networks).

[0101] Step S14, outputting feature information of each feature point according to the feature map that can be directly predicted.

[0102] In this embodiment, the attention mechanism is introduced into the Darknet-53 feature extraction network to increase the weight of the effective feature channels of small head targets in dense crowds and suppress the weight of non-important features, thereby improving the recall rate of small head targets at a bird's-eye view. The recall rate is increased by more than 20% with only a 0.8% increase in the number of parameters and no significant decrease in detection speed.

[0103] Based on the feature points detected by the deep convolutional neural network, in step S2, the feature information of the feature points can be put into a two-dimensional rectangular coordinate system and clustered using a density clustering algorithm (DBSCAN). The density clustering algorithm requires two preset hyperparameters, namely, the neighborhood radius ε and the minimum number of included points MinPts. Figure 5 is an adaptive density clustering method based on device height of an embodiment, such as Figure 5 As shown, the process includes the following steps:

[0104] Step S21, obtain the height information H of the device relative to the ground, and determine the neighborhood radius ε according to the height information H and the preset minimum number of included points MinPts. When MinPts is 10, the relationship between the neighborhood radius ε and the height H of the device can be expressed as follows:

[0105]

[0106] Step S22, obtaining a first feature point cluster with the human head as a feature point, selecting a first feature point that has not been visited in the first feature point cluster, determining adjacent feature points within the neighborhood range of the first feature point based on the neighborhood radius and the positions of other feature points except the first feature point, and generating a second feature point cluster based on the first feature point and the adjacent feature points within the neighborhood range of the first feature point.

[0107] Step S23: determining that the feature points in the first feature point cluster and the second feature point cluster belong to the first category, and determining that the feature points outside the first feature point cluster and the second feature point cluster belong to the second category.

[0108] Step S24, determining that the human body state corresponding to the feature points of the first category is a non-isolated state, and determining that the human body state corresponding to the feature points of the second category is an isolated state.

[0109] This embodiment adopts a highly adaptive parameter adjustment method, fully utilizing the advantages of the density clustering algorithm, and can effectively distinguish between a human body in a non-isolated state and a human body in an isolated state.

[0110] In step S22, it is also determined whether the number of adjacent feature points within the neighborhood of the first feature point reaches the preset minimum number of included points. If it is determined that the number of adjacent feature points within the neighborhood of the first feature point reaches the preset minimum number of included points, a second feature point cluster is generated based on the first feature point and the adjacent feature points within the neighborhood of the first feature point, and the first feature point that has been visited is marked. If it is determined that the number of adjacent feature points within the neighborhood of the first feature point does not reach the preset minimum number of included points, the first feature point is determined to be a noise point, and the first feature point is determined to belong to the second category.

[0111] Among them, the first feature point cluster can be obtained in the following manner: a first feature point is selected from multiple feature points where no feature point has been visited, adjacent feature points within the neighborhood range of the first feature point are determined based on the neighborhood radius and the positions of other feature points except the first feature point, and a first feature point cluster is generated based on the first feature point and the adjacent feature points within the neighborhood range of the first feature point.

[0112] In some embodiments, through recursive step S22, other unvisited points in the cluster are obtained, thereby expanding the cluster. If the cluster is fully expanded, that is, all points in the cluster have been visited, then the above steps S21 to S24 are repeated to visit other unvisited points outside the cluster until all detected feature points of the video frame are included in the visited set. The clustering process can be referred to Figure 6 , Figure 6This is a schematic diagram of the adaptive density clustering results based on device height of this embodiment, wherein the small black dots represent detected feature points, the center of the circle is the visited feature point, the radius of the circle is the neighborhood radius ε, and the arrow directions and connection relationships represent the visit order and access path of the feature points.

[0113] Figure 7 is a flow chart of the human body distribution detection method based on drone of the preferred embodiment, as shown in Figure 7 As shown, the process includes the following steps:

[0114] Step S71: The drone conducts inspection and takes photos to obtain a video stream.

[0115] Step S72, performing human head detection based on the video stream, and using a density clustering algorithm to process feature points, distinguishing human states, and determining a human body in a non-isolated state and a human body in an isolated state.

[0116] Step S73, performing area recognition based on the video stream to determine the safe area and the non-safe area.

[0117] Step S74, determining that a human body in a non-isolated state and in a safe area is a safe object, and determining that a human body in an isolated state and in a non-safe area is a dangerous object.

[0118] Step S75, counting the flow of people at the safe object and obtaining the location of the dangerous object.

[0119] Step S76, integrating the flow of people at the safe object and the location of the dangerous object, and sending the integrated information to the ground command center.

[0120] In combination with the human body distribution detection method of the above-mentioned embodiment, an aerial photography device is also provided in this embodiment, including: a flying body, the flying body equipped with a camera and a processor connected to each other; wherein the camera is used to capture human body images; and the processor is used to execute the human body distribution detection method of any of the above-mentioned embodiments.

[0121] The aerial photography equipment of this embodiment can effectively detect dense crowds and send out alarms when stray individuals enter restricted areas. The aerial photography equipment includes but is not limited to drones and radars with camera and positioning functions.

[0122] In this embodiment, an electronic device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0123] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0124] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0125] S1, obtaining a first video stream collected by a device moving in the air at a bird's-eye view, performing human body detection on the first video stream, and obtaining human body position information of each human body among a plurality of human bodies.

[0126] S2, classifying each human body according to the human body position information of each human body, and determining the human body state of each human body according to the classification result, wherein the human body state includes a non-isolated state and an isolated state.

[0127] S3, obtaining the route information of the device moving in the air, determining the target area in each video frame in the first video stream according to the route information, and determining the human body distribution characteristics in the target area according to the human body position information and human body state of each human body.

[0128] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0129] In addition, in combination with the human body distribution detection method provided in the above embodiments, a storage medium may be provided in this embodiment to implement the method. The storage medium stores a computer program; when the computer program is executed by a processor, any human body distribution detection method in the above embodiments is implemented.

[0130] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0131] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0132] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0133] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A method for detecting human body distribution, characterized in that: include: Acquire a first video stream captured by a device moving in the air at a bird's-eye view angle, perform human body detection on the first video stream, and obtain human body position information of each human body among a plurality of human bodies; Classifying each human body according to the human body position information of each human body, and determining the human body state of each human body according to the classification result, wherein the human body state includes a non-isolated state and an isolated state; Acquire route information of the device moving in the air, determine a target area in each video frame in the first video stream according to the route information, and determine a human body distribution feature in the target area according to the human body position information and the human body state of each human body; Among them, classifying each human body according to the human body position information of each human body, and determining the human body state of each human body according to the classification result includes: obtaining the height information of the device relative to the ground, and determining the neighborhood radius according to the height information and the preset minimum number of included points; obtaining a first feature point cluster with the human head as the feature point, selecting a first feature point that has not been visited in the first feature point cluster, determining the adjacent feature points within the neighborhood range of the first feature point according to the neighborhood radius and the positions of other feature points except the first feature point, and generating a second feature point cluster according to the first feature point and the adjacent feature points within the neighborhood range of the first feature point; determining that the feature points in the first feature point cluster and the second feature point cluster belong to the first category, and determining that the feature points outside the first feature point cluster and the second feature point cluster belong to the second category; determining that the human body state corresponding to the feature points of the first category is the non-isolated state, and determining that the human body state corresponding to the feature points of the second category is the isolated state.

2. The human body distribution detection method according to claim 1, characterized in that: The route information includes a corresponding timestamp and a route position, and determining the target area in each video frame in the first video stream according to the route information includes: Divide the first video stream according to the timestamp to obtain individual video frames; The geographical location corresponding to the target area is determined according to the route position, and the geographical location is mapped to each video frame to obtain the target area in each video frame.

3. The human body distribution detection method according to claim 1, characterized in that: The target area includes a safe area and a non-safe area. After determining the target area in each video frame in the first video stream according to the route information, and determining the human body distribution characteristics in the target area according to the human body position information and the human body state of each human body, the method further includes: Determine that a human body whose state is the non-isolated state and whose position information belongs to the safe area is a safe object; and A human body whose state is the isolated state and whose position information belongs to the unsafe area is determined to be a dangerous object.

4. The human body distribution detection method according to claim 3, characterized in that: The human body is determined to be a safe object when the human body state is the non-isolated state and the human body position information belongs to the safe area; Furthermore, after determining that the human body state is the isolated state and the human body position information belongs to the unsafe area is a dangerous object, the method further includes: Determining the number of the safe objects and obtaining the location information of the dangerous objects; Report the number of the safe objects and the location information of the dangerous objects.

5. The human body distribution detection method according to claim 1, characterized in that: Performing human body detection on the first video stream to obtain human body position information of each of the multiple human bodies includes: The first video stream is input into a trained deep convolutional neural network, target detection is performed on the first video stream with the human head as the target based on the deep convolutional neural network, each of the multiple heads detected is determined as the respective human body, and the position information of the each detected head is determined as the human position information of the respective human body.

6. The human body distribution detection method according to claim 5, characterized in that: Inputting the first video stream into a trained deep convolutional neural network, and performing target detection on the first video stream with the head of a human body as a target based on the deep convolutional neural network includes: Introducing a channel attention mechanism into the deep convolutional neural network, and determining the weight of each feature channel via the channel attention mechanism; According to the weight of each feature channel, the first video stream is downsampled multiple times to obtain a multi-scale feature map with channel attention; The multi-scale feature map is subjected to feature fusion to output feature information of multiple human heads.

7. The human body distribution detection method according to claim 5, characterized in that: Before inputting the first video stream into the trained deep convolutional neural network and performing target detection on the first video stream with the human head as the target based on the deep convolutional neural network, the method further includes: Acquire a second video stream captured by a device moving in the air at a bird's-eye view; Annotating the head of the human body in the second video stream, and dividing the annotated video stream into a training set and a test set according to a preset ratio; Constructing a deep convolutional neural network, wherein the deep convolutional neural network includes N downsampling units, wherein the first M downsampling units are provided with a channel attention unit, and N>M; The deep convolutional neural network is trained according to the training set and the test set to obtain a trained deep convolutional neural network.

8. The human body distribution detection method according to claim 1, characterized in that: Acquiring a first feature point cluster, selecting a first feature point that has not been visited in the first feature point cluster, determining adjacent feature points within a neighborhood range of the first feature point according to the neighborhood radius and positions of other feature points except the first feature point, and generating a second feature point cluster according to the first feature point and adjacent feature points within the neighborhood range of the first feature point, including: Determine whether the number of adjacent feature points within the neighborhood of the first feature point reaches the preset minimum number of included points; When it is determined that the number of adjacent feature points within the neighborhood of the first feature point reaches the preset minimum number of included points, the second feature point cluster is generated based on the first feature point and the adjacent feature points within the neighborhood of the first feature point, and the first feature point that has been visited is marked.

9. The human body distribution detection method according to claim 1, characterized in that: When it is determined that the number of adjacent feature points within the neighborhood of the first feature point does not reach the preset minimum number of included points, the method further includes: The first feature point is determined to be a noise point, and the first feature point is determined to belong to the second category.

10. The human body distribution detection method according to claim 1, characterized in that: Obtaining the first feature point cluster with the human head as the feature point includes: Selecting a first feature point from the plurality of feature points for which no feature point has been accessed; Determine adjacent feature points within the neighborhood range of the first feature point according to the neighborhood radius and positions of other feature points except the first feature point; The first feature point cluster is generated according to the first feature point and adjacent feature points within a neighborhood range of the first feature point.

11. An aerial photography device, characterized in that: include: An aircraft body, wherein the aircraft body is equipped with a camera and a processor connected to each other; wherein the camera is used to shoot a video stream; and the processor is used to execute the human body distribution detection method according to any one of claims 1 to 10.

12. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the human body distribution detection method according to any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the human body distribution detection method according to any one of claims 1 to 10 are implemented.

Images