Passenger flow information generation method and device, electronic equipment and computer readable medium

Abstract

Embodiments of the present disclosure disclose a crowd information generation method and device, electronic equipment and a computer readable medium. A specific embodiment of the method comprises: acquiring a target monitoring scene image; extracting human feature information from the target monitoring scene image; generating a candidate crowd information group according to the human feature information; and generating crowd information corresponding to the target monitoring scene image according to the candidate crowd information group. The embodiment improves the accuracy of identifying the crowd corresponding to the monitoring image.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of computer technology, and more specifically to methods, apparatuses, electronic devices, and computer-readable media for generating pedestrian flow information. Background Technology

[0002] Currently, when identifying pedestrian traffic corresponding to surveillance images, the common methods are: identifying pedestrian traffic by training a regressor that extracts features (such as edges, textures, gradients, etc.) manually from surveillance images to the total number of pedestrians, or identifying pedestrian traffic by training a convolutional neural network that directly extracts the total number of pedestrians from surveillance images.

[0003] However, when using the above methods to identify pedestrian traffic corresponding to surveillance images, the following technical problems often exist: when using a regressor that extracts features manually from surveillance images to the total number of pedestrians to identify pedestrian traffic, there is a lack of high-level semantic information of the images, resulting in low accuracy in identifying pedestrian traffic. When using a convolutional neural network that directly extracts the total number of pedestrians from surveillance images to identify pedestrian traffic, the single-task learning method leads to low accuracy in identifying pedestrian traffic. Summary of the Invention

[0004] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0005] Some embodiments of this disclosure provide methods, apparatuses, electronic devices, and computer-readable media for generating pedestrian flow information to address one or more of the technical problems mentioned in the background section above.

[0006] In a first aspect, some embodiments of this disclosure provide a method for generating pedestrian flow information, the method comprising: acquiring a target monitoring scene image; extracting human feature information from the target monitoring scene image; generating a candidate pedestrian flow information group based on the human feature information; and generating pedestrian flow information corresponding to the target monitoring scene image based on the candidate pedestrian flow information group.

[0007] Optionally, the above-mentioned extraction of human feature information from the target monitoring scene image includes: inputting the target monitoring scene image into a pre-trained human feature extraction network to generate a human feature vector; and inputting the human feature vector into a fully connected layer to generate the human feature information.

[0008] Optionally, generating candidate traffic information groups based on the aforementioned human feature information includes: inputting the aforementioned human feature information into a pre-trained traffic classification network to generate a traffic vector; inputting the aforementioned traffic vector into a first normalization layer to generate a traffic prediction probability vector; and generating classified candidate traffic information based on the aforementioned traffic prediction probability vector.

[0009] Optionally, generating candidate traffic information based on the aforementioned human feature information includes: inputting the aforementioned human feature information into a pre-trained traffic distribution learning network to generate a traffic distribution probability vector; inputting the aforementioned traffic distribution probability vector into a second normalization layer to generate a traffic distribution prediction probability vector; and generating distributed candidate traffic information based on the aforementioned traffic distribution prediction probability vector.

[0010] Optionally, generating candidate traffic information groups based on the aforementioned human feature information further includes: inputting the aforementioned human feature information into a pre-trained traffic regression network to generate regression candidate traffic information.

[0011] Optionally, the aforementioned human feature extraction network includes: a set of convolutional network blocks, wherein the convolutional network blocks in the set of convolutional network blocks are connected in series; and the input of the aforementioned target monitoring scene image into the pre-trained human feature extraction network to generate a human feature vector includes: inputting the aforementioned target monitoring scene image into the convolutional network blocks connected in series in the set of convolutional network blocks to generate a human feature vector.

[0012] Optionally, the convolutional network blocks in the above-mentioned convolutional network block set include: a set of convolutional layers and a set of activation function layers, wherein an activation function layer from the above-mentioned activation function layer set is provided between every two convolutional layers in the above-mentioned convolutional layer set.

[0013] Optionally, generating candidate traffic information based on the aforementioned traffic flow prediction probability vector includes: determining the product of each traffic flow prediction probability in the aforementioned traffic flow prediction probability vector and the target traffic flow as a category traffic flow component, thereby obtaining a category traffic flow component set, wherein the aforementioned target traffic flow corresponds to the aforementioned traffic flow prediction probability; and generating candidate traffic information based on the sum of each category traffic flow component in the aforementioned category traffic flow component set.

[0014] Optionally, generating candidate flow information based on the predicted probability vector of the flow distribution includes: determining the product of each predicted probability of the flow distribution in the predicted probability vector of the flow distribution and the target flow as a flow distribution component, thereby obtaining a set of flow distribution components, wherein the target flow corresponds to the predicted probability of the flow distribution; and generating candidate flow information based on the sum of each flow distribution component in the set of flow distribution components.

[0015] Optionally, the above method further includes: in response to the above-mentioned pedestrian flow information meeting the preset pedestrian flow alarm conditions, controlling the associated alarm device to perform an alarm operation.

[0016] Secondly, some embodiments of this disclosure provide a pedestrian flow information generation device, the device comprising: an acquisition unit configured to acquire a target monitoring scene image; an extraction unit configured to extract human feature information from the target monitoring scene image; a first generation unit configured to generate a candidate pedestrian flow information group based on the human feature information; and a second generation unit configured to generate pedestrian flow information corresponding to the target monitoring scene image based on the candidate pedestrian flow information group.

[0017] Optionally, the extraction unit is further configured to: input the aforementioned target monitoring scene image into a pre-trained human feature extraction network to generate a human feature vector; and input the aforementioned human feature vector into a fully connected layer to generate the aforementioned human feature information.

[0018] Optionally, the first generation unit includes: a first input unit, a second input unit, and a classification candidate traffic information generation unit. The first input unit is configured to input the aforementioned human feature information into a pre-trained traffic classification network to generate a traffic vector. The second input unit is configured to input the aforementioned traffic vector into a first normalization layer to generate a traffic prediction probability vector. The classification candidate traffic information generation unit is configured to generate classification candidate traffic information based on the aforementioned traffic prediction probability vector.

[0019] Optionally, the first generation unit further includes a third input unit, a fourth input unit, and a distribution candidate traffic information generation unit. The third input unit is configured to input the aforementioned human feature information into a pre-trained traffic distribution learning network to generate a traffic distribution probability vector. The fourth input unit is configured to input the aforementioned traffic distribution probability vector into a second normalization layer to generate a traffic distribution prediction probability vector. The distribution candidate traffic information generation unit is configured to generate distribution candidate traffic information based on the aforementioned traffic distribution prediction probability vector.

[0020] Optionally, the first generation unit further includes a fifth input unit, configured to input the aforementioned human feature information into a pre-trained pedestrian flow regression network to generate regression candidate pedestrian flow information.

[0021] Optionally, the aforementioned human feature extraction network includes: a set of convolutional network blocks, wherein the convolutional network blocks in the set of convolutional network blocks are connected in series.

[0022] Optionally, the extraction unit is further configured to input the target monitoring scene image into the convolutional network blocks connected in series in the set of convolutional network blocks to generate human feature vectors.

[0023] Optionally, the convolutional network blocks in the above-mentioned convolutional network block set include: a set of convolutional layers and a set of activation function layers, wherein an activation function layer from the above-mentioned activation function layer set is provided between every two convolutional layers in the above-mentioned convolutional layer set.

[0024] Optionally, the above-mentioned candidate traffic information generation unit is further configured to: determine the product of each traffic prediction probability in the traffic prediction probability vector and the target traffic flow as a traffic flow component, thereby obtaining a traffic flow component set, wherein the target traffic flow corresponds to the traffic prediction probability; and generate candidate traffic information based on the sum of each traffic flow component in the traffic flow component set.

[0025] Optionally, the above-mentioned distribution candidate flow information generation unit is further configured to: determine the product of each flow distribution prediction probability in the above-mentioned flow distribution prediction probability vector and the distribution target flow as a flow distribution component, thereby obtaining a set of flow distribution components, wherein the above-mentioned distribution target flow corresponds to the above-mentioned flow distribution prediction probability; and generate flow distribution candidate information based on the sum of each flow distribution component in the above-mentioned flow distribution component set.

[0026] Optionally, the device further includes a control unit configured to control associated alarm devices to perform alarm operations in response to the aforementioned pedestrian flow information meeting preset pedestrian flow alarm conditions.

[0027] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation of the first aspect above.

[0028] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect above.

[0029] The above embodiments of this disclosure have the following beneficial effects: the pedestrian flow information generation method of some embodiments of this disclosure improves the accuracy of identifying pedestrian flow corresponding to surveillance images. Specifically, the reason for the low accuracy of identifying pedestrian flow corresponding to surveillance images is that when using a regressor to identify pedestrian flow from manually extracted features of the surveillance image to the total number of pedestrians, there is a lack of high-level semantic information of the image, resulting in low accuracy of pedestrian flow identification. When using a convolutional neural network to identify pedestrian flow directly from the surveillance image to the total number of pedestrians, the single-task learning method leads to low accuracy of pedestrian flow identification. Based on this, the pedestrian flow information generation method of some embodiments of this disclosure first acquires a target surveillance scene image. Then, human feature information is extracted from the target surveillance scene image. Thus, the automatically extracted human feature information contains high-level semantic information of the target surveillance scene image. Afterwards, a candidate pedestrian flow information group is generated based on the human feature information. Finally, pedestrian flow information corresponding to the target surveillance scene image is generated based on the candidate pedestrian flow information group. Because the pedestrian flow information is generated from each candidate pedestrian flow information in the candidate pedestrian flow information group, a multi-task learning method is used, which improves the accuracy of identifying pedestrian flow corresponding to surveillance images. Attached Figure Description

[0030] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0031] Figure 1 This is a schematic diagram illustrating an application scenario of a method for generating pedestrian flow information according to some embodiments of this disclosure;

[0032] Figure 2 This is a flowchart of some embodiments of the method for generating pedestrian flow information according to this disclosure;

[0033] Figure 3 This is a flowchart of some other embodiments of the method for generating pedestrian flow information according to this disclosure;

[0034] Figure 4 It is the network structure and data flow diagram of the network for generating pedestrian flow information;

[0035] Figure 5 This is a schematic diagram of the network structure of a human feature extraction network;

[0036] Figure 6 This is a schematic diagram of the network structure of a convolutional network block;

[0037] Figure 7These are schematic diagrams illustrating the structure of some embodiments of the pedestrian flow information generation device according to this disclosure;

[0038] Figure 8 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0039] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0040] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0041] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0042] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0043] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0044] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0045] Figure 1 This is a schematic diagram illustrating an application scenario of a method for generating pedestrian flow information according to some embodiments of this disclosure.

[0046] exist Figure 1In the application scenario, firstly, the computing device 101 can acquire the target monitoring scene image 102. Then, the computing device 101 can extract human feature information 103 from the target monitoring scene image 102. Afterwards, the computing device 101 can generate a candidate flow information group 104 based on the human feature information 103 (for example, the candidate flow information group 104 may include: 8, 9, 10). Each candidate flow information can represent the number of people in the target monitoring scene image 102, i.e., the flow of people. Finally, the computing device 101 can generate the flow of people information 105 corresponding to the target monitoring scene image 102 based on the candidate flow information group 104 (for example, the flow of people information 105 can be 9).

[0047] It should be noted that the aforementioned computing device 101 can be either hardware or software. When the computing device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device. When the computing device is software, it can be installed within the hardware devices listed above. It can be implemented as, for example, multiple software programs or software modules used to provide distributed services, or as a single software program or software module. No specific limitations are made here.

[0048] It should be understood that Figure 1 The number of computing devices shown is merely illustrative. Any number of computing devices can be used depending on implementation needs.

[0049] Continue to refer to Figure 2 The flowchart 200 illustrates some embodiments of the pedestrian flow information generation method according to the present disclosure. The pedestrian flow information generation method includes the following steps:

[0050] Step 201: Obtain the target monitoring scene image.

[0051] In some embodiments, the entity executing the pedestrian flow information generation method (e.g.) Figure 1The computing device 101 shown can acquire images of the target monitoring scene via a wired or wireless connection. The target monitoring scene image can be an image captured by a camera device in a currently acquired monitoring scene. The monitoring scene can be a scene with crowd movement. In practice, the executing entity can retrieve a monitoring scene image with a preset capture time from a database storing images of various monitoring scenes as the target monitoring scene image. The preset time can be the current time or a historical time. It should be noted that the wireless connection method can include, but is not limited to, 3G / 4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra-wideband) connection, and other currently known or future known wireless connection methods.

[0052] Step 202: Extract human feature information from the target monitoring scene image.

[0053] In some embodiments, the execution entity can extract human feature information from the target surveillance scene image. In practice, the execution entity can extract human feature information from the target surveillance scene image using a human feature extraction model. The human feature information can be information containing human features included in the target surveillance scene image. The human feature extraction model can be a model used to extract human features included in the target surveillance scene image. For example, the human feature extraction model can be, but is not limited to, one of the following: RCNN (Region Convolutional Neural Networks) model, or DCNN (Deep Convolutional Neural Networks) model.

[0054] Step 203: Generate candidate traffic information groups based on human body feature information.

[0055] In some embodiments, the executing entity can generate a candidate flow information group based on the aforementioned human feature information. In practice, the executing entity can generate flow as candidate flow information based on each flow recognition algorithm in a preset flow recognition algorithm set and the aforementioned human feature information. The flow recognition algorithms in the aforementioned flow recognition algorithm set can be algorithms used to identify flow. For example, the aforementioned flow recognition algorithm set may include, but is not limited to, decision trees and support vector machines. The aforementioned flow refers to the number of people in the target monitoring scene image identified by the flow recognition algorithm.

[0056] Step 204: Generate pedestrian flow information corresponding to the target monitoring scene image based on the candidate flow information group.

[0057] In some embodiments, the executing entity can generate pedestrian flow information corresponding to the target monitoring scene image based on the candidate pedestrian flow information group. In practice, the executing entity can determine the average value of the pedestrian flow information of each candidate included in the candidate pedestrian flow information group as the pedestrian flow information corresponding to the target monitoring scene image.

[0058] Optionally, the aforementioned executing entity may, in response to the aforementioned pedestrian flow information meeting a preset pedestrian flow alarm condition, control the associated alarm device to perform an alarm operation. The preset pedestrian flow alarm condition may be "the pedestrian flow corresponding to the pedestrian flow information is greater than a preset value." The specific setting of the preset value is not limited here. The aforementioned alarm device may be a speaker. The aforementioned alarm operation may be the operation of the aforementioned alarm device to emit a preset alarm prompt sound.

[0059] The above embodiments of this disclosure have the following beneficial effects: the pedestrian flow information generation method of some embodiments of this disclosure improves the accuracy of identifying pedestrian flow corresponding to surveillance images. Specifically, the reason for the low accuracy of identifying pedestrian flow corresponding to surveillance images is that when using a regressor to identify pedestrian flow from manually extracted features of the surveillance image to the total number of pedestrians, there is a lack of high-level semantic information of the image, resulting in low accuracy of pedestrian flow identification. When using a convolutional neural network to identify pedestrian flow directly from the surveillance image to the total number of pedestrians, the single-task learning method leads to low accuracy of pedestrian flow identification. Based on this, the pedestrian flow information generation method of some embodiments of this disclosure first acquires a target surveillance scene image. Then, human feature information is extracted from the target surveillance scene image. Thus, the automatically extracted human feature information contains high-level semantic information of the target surveillance scene image. Afterwards, a candidate pedestrian flow information group is generated based on the human feature information. Finally, pedestrian flow information corresponding to the target surveillance scene image is generated based on the candidate pedestrian flow information group. Because the pedestrian flow information is generated from each candidate pedestrian flow information in the candidate pedestrian flow information group, a multi-task learning method is used, which improves the accuracy of identifying pedestrian flow corresponding to surveillance images.

[0060] Further reference Figure 3 This illustrates a flow 300 of another embodiment of the method for generating pedestrian flow information. Flow 300 of this method includes the following steps:

[0061] Step 301: Obtain the target monitoring scene image.

[0062] In some embodiments, the specific implementation of step 301 and its resulting technical effects can be found in [reference needed]. Figure 2 Step 201 in the corresponding embodiments will not be repeated here.

[0063] Step 302: Input the target monitoring scene image into a pre-trained human feature extraction network to generate a human feature vector.

[0064] In some embodiments, the entity executing the pedestrian flow information generation method (e.g.) Figure 1 The computing device 101 shown can input the aforementioned target surveillance scene image into a pre-trained human feature extraction network to generate a human feature vector. The aforementioned human feature extraction network can be a network used to extract human features contained in the aforementioned target surveillance scene image. For example, the aforementioned human feature extraction network can be, but is not limited to, any of the following: a VGG16 (Visual Geometry Group-16) network and a VGG32 (Visual Geometry Group-32) network.

[0065] As an example, such as Figure 4 The diagram shows the network structure and data flow of the pedestrian flow information generation network. This network includes: a human feature extraction network 401, a fully connected layer 402, a pedestrian flow classification network 403, a first normalization layer 404, a pedestrian flow distribution learning network 405, a second normalization layer 406, and a pedestrian flow regression network 407. The executing entity can input the target monitoring scene image 102 into the human feature extraction network 401 to generate a human feature vector 408.

[0066] Optionally, the aforementioned human feature extraction network may include: a set of convolutional network blocks, wherein the convolutional network blocks in the set of convolutional network blocks are connected in series.

[0067] In some optional implementations of certain embodiments, the execution entity may input the target monitoring scene image into the serially connected convolutional network blocks in the set of convolutional network blocks to generate human feature vectors.

[0068] As an example, such as Figure 5 The diagram shows the network structure of the human feature extraction network. The human feature extraction network 401 may include at least one convolutional network block 501. The convolutional network blocks in the at least one convolutional network block 501 are connected serially.

[0069] Optionally, the convolutional network blocks in the above-mentioned set of convolutional network blocks may include: a set of convolutional layers and a set of activation function layers, wherein an activation function layer from the above-mentioned set of activation function layers is provided between every two convolutional layers in the above-mentioned set of convolutional layers.

[0070] As an example, such as Figure 6The diagram shows the network structure of a convolutional network block 501, which includes multiple convolutional layers 601 and multiple activation function layers 602. Each convolutional layer 601 is followed by an activation function layer 602.

[0071] Step 303: Input the human feature vector into the fully connected layer to generate human feature information.

[0072] In some embodiments, the execution entity can input the human feature vector into a fully connected layer to generate the human feature information. For example, the execution entity can input the human feature vector 408 into the fully connected layer 402 to generate the human feature information 103. The human feature information can be represented by a feature vector. The human feature information can be represented by f, where f ∈ R. c Where R represents the vector space, and c represents the vector length of the aforementioned human feature information.

[0073] Step 304: Input human feature information into a pre-trained pedestrian flow classification network to generate a pedestrian flow vector.

[0074] In some embodiments, the aforementioned execution entity may input the aforementioned human feature information into a pre-trained pedestrian flow classification network to generate a pedestrian flow vector. For example, such as... Figure 4 As shown, the aforementioned execution entity can input the aforementioned human feature information 103 into a pre-trained pedestrian flow classification network 403 to generate a pedestrian flow vector 409. The aforementioned pedestrian flow classification network can be a fully connected layer.

[0075] Among them, the above-mentioned pedestrian flow vector can be used express. Here, R represents the vector space. M+1 represents the M+1 classes from 0 to M.

[0076] Step 305: Input the pedestrian flow vector into the first normalization layer to generate a pedestrian flow prediction probability vector.

[0077] In some embodiments, the execution entity may input the pedestrian flow vector into a first normalization layer to generate a pedestrian flow prediction probability vector. The first normalization layer may be a normalization layer corresponding to the pedestrian flow classification network. The pedestrian flow prediction probability in the pedestrian flow prediction probability vector can characterize the predicted probability of the pedestrian flow corresponding to the pedestrian flow vector.

[0078] As an example, such as Figure 4 As shown, the aforementioned execution entity can input the aforementioned pedestrian flow vector 409 into the aforementioned first normalization layer 404 to generate a pedestrian flow prediction probability vector 410.

[0079] Among them, the above-mentioned probability vector for predicting pedestrian flow can be used express. and Represent and The j-th element.

[0080] Step 306: Generate traffic information for categorized candidates based on the predicted probability vector of pedestrian traffic.

[0081] In some embodiments, the executing entity can generate classification candidate traffic information based on the aforementioned traffic flow prediction probability vector. In practice, the executing entity can use the traffic flow corresponding to the highest traffic flow prediction probability in the aforementioned traffic flow prediction probability vector as the classification candidate traffic information. For example, such as... Figure 4 As shown, the aforementioned executing entity can determine the traffic flow corresponding to the largest traffic flow prediction probability in the traffic flow prediction probability vector 410 as the traffic flow information 411 for the classification candidate.

[0082] In some optional implementations of certain embodiments, firstly, the executing entity can determine the product of each predicted pedestrian flow probability in the predicted pedestrian flow probability vector and the target pedestrian flow as a category pedestrian flow component, thus obtaining a set of category pedestrian flow components. The target pedestrian flow corresponds to the predicted pedestrian flow probability. Then, category candidate pedestrian flow information can be generated based on the sum of each category pedestrian flow component in the set of category pedestrian flow components. In practice, the executing entity can determine the category candidate pedestrian flow information by the sum of each category pedestrian flow component.

[0083] Step 307: Input human feature information into a pre-trained pedestrian flow distribution learning network to generate a pedestrian flow distribution probability vector.

[0084] In some embodiments, the aforementioned execution entity may input the aforementioned human feature information into a pre-trained pedestrian flow distribution learning network to generate a pedestrian flow distribution probability vector.

[0085] As an example, such as Figure 4 As shown, the aforementioned execution entity can input the aforementioned human feature information 103 into a pre-trained pedestrian flow distribution learning network 405 to generate a pedestrian flow distribution probability vector 412. The aforementioned pedestrian flow distribution learning network can be a fully connected layer. The aforementioned pedestrian flow distribution probability vector can be... express. The aforementioned pedestrian flow distribution learning network can be a fully connected layer.

[0086] Step 308: Input the pedestrian flow distribution probability vector into the second normalization layer to generate the pedestrian flow distribution prediction probability vector.

[0087] In some embodiments, the execution entity may input the pedestrian flow distribution probability vector into a second normalization layer to generate a pedestrian flow distribution prediction probability vector. The second normalization layer may be a normalization layer corresponding to the pedestrian flow distribution learning network. The pedestrian flow distribution prediction probability in the pedestrian flow distribution prediction probability vector can characterize the predicted probability of the pedestrian flow distribution probability corresponding to the pedestrian flow distribution probability vector.

[0088] As an example, such as Figure 4 As shown, the aforementioned execution entity can input the aforementioned pedestrian flow distribution probability vector 412 into the aforementioned second normalization layer 406 to generate a pedestrian flow distribution prediction probability vector 413.

[0089] Among them, the above-mentioned probability vector for predicting pedestrian flow distribution can be used as follows: express. and Represent and The j-th element.

[0090] Step 309: Generate distribution candidate traffic information based on the probability vector predicted by the traffic flow distribution.

[0091] In some embodiments, the executing entity can generate candidate flow information based on the predicted flow probability vector. In practice, the executing entity can determine the flow rate corresponding to the highest predicted flow probability in the predicted flow probability vector as the candidate flow information. For example, ... Figure 4 As shown, the aforementioned executing entity can determine the traffic flow corresponding to the largest traffic flow distribution prediction probability in the traffic flow distribution prediction probability vector 413 as the traffic flow distribution candidate information 414.

[0092] In some optional implementations of certain embodiments, firstly, the executing entity can determine the distributed pedestrian flow component by multiplying the predicted pedestrian flow probability of each individual pedestrian flow distribution in the predicted pedestrian flow distribution probability vector with the target pedestrian flow, thus obtaining a set of distributed pedestrian flow components. The target pedestrian flow corresponds to the predicted pedestrian flow distribution probability. Then, distributed candidate pedestrian flow information can be generated based on the sum of each distributed pedestrian flow component in the set of distributed pedestrian flow components. In practice, the executing entity can determine the distributed candidate pedestrian flow information by determining the sum of each distributed pedestrian flow component.

[0093] Step 310: Input human feature information into a pre-trained pedestrian flow regression network to generate regression candidate flow information.

[0094] In some embodiments, the aforementioned execution entity may input the aforementioned human feature information into a pre-trained pedestrian flow regression network to generate regression candidate pedestrian flow information. The aforementioned pedestrian flow regression network may include fully connected layers. For example, such as... Figure 4 As shown, the aforementioned executing entity can input the aforementioned human feature information 103 into the pre-trained pedestrian flow regression network 407 to generate regression candidate flow information 415. The regression candidate flow information can be represented by y3.

[0095] As an example, such as Figure 4 As shown, the aforementioned pedestrian flow information generation network can be trained through the following steps:

[0096] The first step is to obtain a set of surveillance scene image samples. This set includes human images and pedestrian flow labels. The pedestrian flow labels can be the average number of people labeled by n annotators for a given surveillance scene image sample. The minimum number of people in each surveillance scene image sample is 0, and the maximum number is M.

[0097] The second step is to generate a pedestrian flow distribution probability label for each monitoring scene image sample in the aforementioned monitoring scene image sample set. This pedestrian flow distribution probability label can be represented by the denoting character p.

[0098] In practice, the aforementioned implementing entity can generate a pedestrian flow distribution probability label p in one dimension using the following formula. j :

[0099]

[0100] Where, p j Let p represent the probability that the number of people in the above surveillance scene image sample is j. σ represents the standard deviation of the Gaussian distribution. Z is a normalization factor to ensure that the sum of the dimensions of p is 1. y represents the pedestrian flow label of the image sample in the above monitoring scene. Representing e The power of e. e is the natural base.

[0101] The third step is to divide the above-mentioned set of monitoring scene image samples according to a preset ratio to generate a training sample set and a test sample set.

[0102] For example, the ratio of the number of training samples in the training sample set to the number of test samples in the test sample set could be 10:1.

[0103] The fourth step is to randomize the pedestrian flow information to generate network parameters.

[0104] The fifth step is to train the above-mentioned pedestrian flow information generation network using the aforementioned training sample set.

[0105] Alternatively, the following formula can be used as the loss function for the above-mentioned pedestrian flow classification network:

[0106]

[0107] Where L1 represents the loss value of the loss function of the pedestrian flow classification network. y represents the pedestrian flow labels included in the training samples.

[0108] Alternatively, the following formula can be used as the loss function for the above-mentioned pedestrian flow distribution learning network:

[0109]

[0110] Where L2 represents the loss value of the loss function of the above-mentioned pedestrian flow distribution learning network.

[0111] Alternatively, the following formula can be used as the loss function for the aforementioned pedestrian flow regression network:

[0112] L3 = |y3 - y|.

[0113] Where L3 represents the loss value of the loss function of the aforementioned pedestrian flow regression network.

[0114] Alternatively, the following formula can be used as the ensemble loss function for the aforementioned pedestrian flow regression network:

[0115]

[0116] Where L4 represents the loss value of the loss function of the aforementioned pedestrian flow regression network. This represents the integrated output of the pedestrian flow classification network, pedestrian flow distribution learning network, and pedestrian flow regression network corresponding to the training samples.

[0117] Optionally, Figure 4 The overall loss function of the pedestrian flow information generation network shown can be:

[0118] L = L1 + L2 + L3 + L4.

[0119] Where L represents the loss value of the overall loss function.

[0120] The sixth step is to update the network parameters of the pedestrian flow information generation network using the backpropagation algorithm.

[0121] The learning rate of the backpropagation algorithm can be a preset value. For example, the learning rate can be 0.01. Alternatively, the learning rate can also be 0.001.

[0122] The seventh step is to test the trained pedestrian flow information generation network using a test sample set.

[0123] Step 311: Generate pedestrian flow information corresponding to the target monitoring scene image based on the candidate flow information group.

[0124] In some embodiments, firstly, the executing entity can combine the categorized candidate traffic information, the distributed candidate traffic information, and the regression candidate traffic information into a candidate traffic information group. Then, the executing entity can determine the average of the categorized candidate traffic information, the distributed candidate traffic information, and the regression candidate traffic information as a traffic volume value. Next, the traffic volume value can be rounded down to obtain the rounded traffic volume value as the traffic volume information.

[0125] from Figure 3 It can be seen from this that, with Figure 2 Compared to the description of some corresponding embodiments, Figure 3 In some corresponding embodiments, the process 300 of the pedestrian flow information generation method firstly extracts human features through a human feature extraction network and a fully connected layer, thereby extracting deeper human feature information. Secondly, it generates classification candidate pedestrian flow information, distribution candidate pedestrian flow information, and regression candidate pedestrian flow information through a pedestrian flow classification network, a pedestrian flow distribution learning network, and a pedestrian flow regression network, respectively. This allows the pedestrian flow classification network, the pedestrian flow distribution learning network, and the pedestrian flow regression network to complement each other, improving the accuracy of the obtained pedestrian flow information.

[0126] Further reference Figure 7 As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of a pedestrian flow information generation device, which are similar to... Figure 2 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0127] like Figure 7As shown, a pedestrian flow information generation device 700 in some embodiments includes: an acquisition unit 701, an extraction unit 702, a first generation unit 703, and a second generation unit 704. The acquisition unit 701 is configured to acquire a target monitoring scene image; the extraction unit 702 is configured to extract human feature information from the target monitoring scene image; the first generation unit 703 is configured to generate a candidate pedestrian flow information group based on the human feature information; and the second generation unit 704 is configured to generate pedestrian flow information corresponding to the target monitoring scene image based on the candidate pedestrian flow information group.

[0128] Optionally, the extraction unit can be further configured to: input the aforementioned target monitoring scene image into a pre-trained human feature extraction network to generate a human feature vector; and input the aforementioned human feature vector into a fully connected layer to generate the aforementioned human feature information.

[0129] Optionally, the first generation unit may include: a first input unit, a second input unit, and a classification candidate traffic information generation unit (not shown in the figure). The first input unit is configured to input the aforementioned human feature information into a pre-trained traffic classification network to generate a traffic vector. The second input unit is configured to input the aforementioned traffic vector into a first normalization layer to generate a traffic prediction probability vector. The classification candidate traffic information generation unit is configured to generate classification candidate traffic information based on the aforementioned traffic prediction probability vector.

[0130] Optionally, the first generation unit may further include: a third input unit, a fourth input unit, and a distribution candidate traffic information generation unit (not shown in the figure). The third input unit is configured to input the aforementioned human feature information into a pre-trained traffic distribution learning network to generate a traffic distribution probability vector. The fourth input unit is configured to input the aforementioned traffic distribution probability vector into a second normalization layer to generate a traffic distribution prediction probability vector. The distribution candidate traffic information generation unit is configured to generate distribution candidate traffic information based on the aforementioned traffic distribution prediction probability vector.

[0131] Optionally, the first generation unit may further include a fifth input unit (not shown in the figure), configured to input the above-mentioned human feature information into a pre-trained pedestrian flow regression network to generate regression candidate pedestrian flow information.

[0132] Optionally, the aforementioned human feature extraction network includes: a set of convolutional network blocks, wherein the convolutional network blocks in the set of convolutional network blocks are connected in series.

[0133] Optionally, the extraction unit can be further configured to input the target monitoring scene image into the convolutional network blocks connected in series in the set of convolutional network blocks to generate human feature vectors.

[0134] Optionally, the convolutional network blocks in the above-mentioned convolutional network block set include: a set of convolutional layers and a set of activation function layers, wherein an activation function layer from the above-mentioned activation function layer set is provided between every two convolutional layers in the above-mentioned convolutional layer set.

[0135] Optionally, the above-mentioned candidate traffic information generation unit can be further configured to: determine the product of each traffic prediction probability in the traffic prediction probability vector and the target traffic flow as a traffic flow component, thereby obtaining a traffic flow component set, wherein the target traffic flow corresponds to the traffic prediction probability; and generate candidate traffic information based on the sum of each traffic flow component in the traffic flow component set.

[0136] Optionally, the above-mentioned distribution candidate flow information generation unit can be further configured to: determine the product of each flow distribution prediction probability in the above-mentioned flow distribution prediction probability vector and the distribution target flow as a flow distribution component, thereby obtaining a set of flow distribution components, wherein the above-mentioned distribution target flow corresponds to the above-mentioned flow distribution prediction probability; and generate flow distribution candidate information based on the sum of each flow distribution component in the above-mentioned flow distribution component set.

[0137] Optionally, the pedestrian flow information generation device 700 may further include: a control unit (not shown in the figure), configured to control an associated alarm device to perform an alarm operation in response to the pedestrian flow information meeting a preset pedestrian flow alarm condition.

[0138] It is understandable that the units described in the device 700 are related to the reference. Figure 2 The steps in the described method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method also apply to the device 700 and the units contained therein, and will not be repeated here.

[0139] The following is for reference. Figure 8 It illustrates electronic devices suitable for implementing some embodiments of this disclosure (e.g., Figure 1 A schematic diagram of the structure of the computing device 101)800. Figure 8 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0140] like Figure 8As shown, the electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 into a random access memory (RAM) 803. The RAM 803 also stores various programs and data required for the operation of the electronic device 800. The processing device 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0141] Typically, the following devices can be connected to I / O interface 805: input devices 806 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 807 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 808 including, for example, magnetic tapes, hard disks, etc.; and communication devices 809. Communication device 809 allows electronic device 800 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 8 An electronic device 800 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 8 Each box shown can represent a device or multiple devices as needed.

[0142] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 809, or installed from a storage device 808, or installed from a ROM 802. When the computer program is executed by the processing device 801, it performs the functions defined in the methods of some embodiments of this disclosure.

[0143] It should be noted that, in some embodiments of this disclosure, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0144] In some implementations, clients and servers may communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and may interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0145] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: acquire a target monitoring scene image; extract human feature information from the target monitoring scene image; generate a candidate traffic information group based on the human feature information; and generate pedestrian traffic information corresponding to the target monitoring scene image based on the candidate traffic information group.

[0146] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0147] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0148] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including an acquisition unit, an extraction unit, a first generation unit, and a second generation unit. The names of these units do not necessarily limit the specific unit; for example, the acquisition unit may also be described as "a unit for acquiring images of a target monitoring scene."

[0149] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0150] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A method for generating pedestrian flow information, comprising: Acquire images of the target monitoring scene; Extract human feature information from the target surveillance scene image; Based on the aforementioned human characteristic information, a candidate traffic information group is generated, including: The human feature information is input into a pre-trained pedestrian flow classification network to generate a pedestrian flow vector; The pedestrian flow vector is input into the first normalization layer to generate a pedestrian flow prediction probability vector. Based on the predicted pedestrian flow probability vector, generate classified candidate pedestrian flow information; The human feature information is input into a pre-trained pedestrian flow distribution learning network to generate a pedestrian flow distribution probability vector; The probability vector of pedestrian flow distribution is input into the second normalization layer to generate a predicted probability vector of pedestrian flow distribution. Based on the predicted probability vector of pedestrian flow distribution, generate distribution candidate flow information; The human feature information is input into a pre-trained pedestrian flow regression network to generate regression candidate pedestrian flow information; The mean of the classified candidate traffic information, the distributed candidate traffic information, and the regressed candidate traffic information is determined as the traffic volume value. The traffic volume value is then rounded down to obtain the traffic volume value after rounding down, which is used as the traffic volume information corresponding to the target monitoring scene image.

2. The method according to claim 1, wherein, The step of extracting human feature information from the target surveillance scene image includes: The target monitoring scene image is input into a pre-trained human feature extraction network to generate a human feature vector; The human feature vector is input into a fully connected layer to generate the human feature information.

3. The method according to claim 2, wherein, The human feature extraction network includes: a set of convolutional network blocks, wherein the convolutional network blocks in the set of convolutional network blocks are connected sequentially; and The step of inputting the target surveillance scene image into a pre-trained human feature extraction network to generate a human feature vector includes: The target monitoring scene image is input into a series of convolutional network blocks in the set of convolutional network blocks to generate a human feature vector.

4. The method according to claim 3, wherein, The convolutional network blocks in the set of convolutional network blocks include: a set of convolutional layers and a set of activation function layers, wherein an activation function layer from the set of activation function layers is set between every two convolutional layers in the set of convolutional layers.

5. The method according to claim 1, wherein, The step of generating classification candidate traffic information based on the predicted pedestrian flow probability vector includes: The product of each predicted pedestrian flow probability in the predicted pedestrian flow probability vector and the target pedestrian flow is determined as the target pedestrian flow component, thus obtaining the set of target pedestrian flow components, wherein the target pedestrian flow corresponds to the predicted pedestrian flow probability. Based on the sum of the traffic flow components of each category in the set of categorized traffic flow components, categorized candidate traffic information is generated.

6. The method according to claim 1, wherein, The step of generating distribution candidate traffic information based on the predicted probability vector of the traffic flow distribution includes: The product of each predicted pedestrian flow distribution probability in the predicted pedestrian flow distribution vector and the target pedestrian flow distribution is determined as a pedestrian flow distribution component, thus obtaining a set of pedestrian flow distribution components, wherein the target pedestrian flow distribution corresponds to the predicted pedestrian flow distribution probability. Based on the sum of each distributed pedestrian flow component in the distributed pedestrian flow component set, distributed candidate flow information is generated.

7. The method according to any one of claims 1-6, wherein, The method further includes: In response to the pedestrian flow information meeting the preset pedestrian flow alarm conditions, the associated alarm device is controlled to perform an alarm operation.

8. A device for generating pedestrian flow information, comprising: The acquisition unit is configured to acquire images of the target monitored scene. The extraction unit is configured to extract human feature information from the target surveillance scene image. The first generation unit is configured to generate a candidate traffic information group based on the human body feature information, including: The human feature information is input into a pre-trained pedestrian flow classification network to generate a pedestrian flow vector; The pedestrian flow vector is input into the first normalization layer to generate a pedestrian flow prediction probability vector. Based on the predicted pedestrian flow probability vector, generate classified candidate pedestrian flow information; The human feature information is input into a pre-trained pedestrian flow distribution learning network to generate a pedestrian flow distribution probability vector; The probability vector of pedestrian flow distribution is input into the second normalization layer to generate a predicted probability vector of pedestrian flow distribution. Based on the predicted probability vector of pedestrian flow distribution, generate distribution candidate flow information; The human feature information is input into a pre-trained pedestrian flow regression network to generate regression candidate pedestrian flow information; The second generation unit is configured to determine the average of the classification candidate traffic information, the distribution candidate traffic information, and the regression candidate traffic information as the traffic value, and to round down the traffic value to obtain the rounded traffic value as the traffic information corresponding to the target monitoring scene image.

9. An electronic device, comprising: One or more processors; A storage device on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1 to 7.

10. A computer-readable medium having a computer program stored thereon, wherein, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 7.

Images