A passenger flow counting method, device, equipment and medium based on image generation

Abstract

The present application discloses a passenger flow counting method, device, equipment and medium based on image generation, which relates to the field of passenger flow counting. The method comprises: detecting and counting complete pedestrians in a passenger flow area image to obtain a foreground passenger flow; removing complete pedestrians in the passenger flow area image to obtain an occluded image; detecting occluded objects in the occluded image, and removing the occluded objects in the occluded image to obtain a remaining image area; inputting pixel values ​​in the remaining image area into an image generation model, reconstructing the area where the occluded objects are located, and determining the occluded image after the occluded objects are reconstructed as a reconstructed image; the image generation model is constructed based on a deep learning model; detecting and counting complete pedestrians in the reconstructed image to obtain the passenger flow in the occluded area; determining the final passenger flow in the target scene according to the foreground passenger flow and the passenger flow in the occluded area. The present application can effectively solve the occlusion problem and improve the accuracy of passenger flow counting.

Description

Technical Field

[0001] The present application relates to the field of passenger flow statistics, and in particular to a passenger flow statistics method, device, equipment and medium based on image generation. Background Art

[0002] With the continuous progress of society, the flow of people in cities, especially in buses, subways, airports, railway stations, large shopping malls, shopping centers, chain stores, etc. is increasing. In order to provide effective personnel management data information for large commercial platforms or human resource management platforms in a timely manner, it is usually necessary to count or record the passenger flow.

[0003] The commonly used passenger flow counting method at present is to directly count the people in the scene image. However, due to factors such as object occlusion, this method often leads to inaccurate statistics, affecting the analysis of passenger flow data. Summary of the invention

[0004] The purpose of this application is to provide a passenger flow counting method, device, equipment and medium based on image generation, which can effectively solve the occlusion problem and improve the accuracy of passenger flow counting.

[0005] To achieve the above objectives, this application provides the following solutions.

[0006] In a first aspect, the present application provides a passenger flow counting method based on image generation, and the passenger flow counting method based on image generation includes: obtaining a passenger flow area image in a target scene; detecting and counting complete pedestrians in the passenger flow area image to obtain a foreground passenger flow; removing complete pedestrians in the passenger flow area image to obtain an occluded image; detecting occluded objects in the occluded image, and removing the occluded objects in the occluded image to obtain a remaining image area; inputting pixel values ​​in the remaining image area into an image generation model, reconstructing the area where the occluded object is located, and determining the occluded image after the occluded object is reconstructed as a reconstructed image; the image generation model is constructed based on a deep learning model; detecting and counting complete pedestrians in the reconstructed image to obtain the passenger flow in the occluded area; determining the final passenger flow in the target scene based on the foreground passenger flow and the passenger flow in the occluded area.

[0007] In a second aspect, the present application provides a passenger flow counting device based on image generation, and the passenger flow counting device based on image generation includes: an image acquisition module, which is used to acquire a passenger flow area image in a target scene; a foreground counting module, which is used to detect and count complete pedestrians in the passenger flow area image to obtain a foreground passenger flow; a pedestrian removal module, which is used to remove complete pedestrians in the passenger flow area image to obtain an occlusion image; an object detection and removal module, which is used to detect occluding objects in the occluding image and remove the occluding objects in the occluding image to obtain a remaining image area; an image reconstruction module, which is used to input pixel values ​​in the remaining image area into an image generation model, reconstruct the area where the occluding object is located, and determine the occluding image after the occluding object is reconstructed as a reconstructed image; the image generation model is constructed based on a deep learning model; an occlusion area counting module, which is used to detect and count complete pedestrians in the reconstructed image to obtain the passenger flow in the occlusion area; a passenger flow determination module, which is used to determine the final passenger flow in the target scene according to the foreground passenger flow and the occlusion area passenger flow.

[0008] In a third aspect, the present application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the passenger flow counting method based on image generation described in the first aspect above.

[0009] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the passenger flow counting method based on image generation described in the first aspect above.

[0010] According to the specific embodiments provided by the present application, the present application discloses the following technical effects: the present application provides a passenger flow statistics method, device, equipment and medium based on image generation, takes into account the object occlusion factor, constructs an image generation model based on a deep learning model, and uses the image generation model to realize the reconstruction of the area where the occluded objects are located in the passenger flow area image, and completes the statistics of the passenger flow in the area where the occluded objects are located in the passenger flow area image, effectively solving the occlusion problem and improving the accuracy of passenger flow statistics. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0012] Figure 1This is a diagram of the application environment of a passenger flow statistics method based on image generation in one embodiment of the present application.

[0013] Figure 2 A flowchart of a passenger flow counting method based on image generation provided in one embodiment of the present application.

[0014] Figure 3 A schematic diagram of functional modules of a passenger flow counting device based on image generation provided in one embodiment of the present application.

[0015] Figure 4 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0017] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0018] The passenger flow counting method based on image generation provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the passenger flow area image in the target scene to the server 104. After the server 104 receives the passenger flow area image in the target scene, for the passenger flow area image, the server 104 detects and counts the complete pedestrians in the passenger flow area image to obtain the foreground passenger flow; removes the complete pedestrians in the passenger flow area image to obtain the occlusion image; detects the occlusion object in the occlusion image, and removes the occlusion object in the occlusion image to obtain the remaining image area; inputs the pixel values ​​in the remaining image area into the image generation model, reconstructs the area where the occlusion object is located, and determines the occlusion image after the occlusion object is reconstructed as the reconstructed image; the image generation model is constructed based on the deep learning model; detects and counts the complete pedestrians in the reconstructed image to obtain the passenger flow in the occlusion area; determines the final passenger flow in the target scene according to the foreground passenger flow and the passenger flow in the occlusion area. The server 104 may feed back the final passenger flow in the target scene to the terminal 102 .

[0019] In addition, in some embodiments, the passenger flow statistics method based on image generation can also be implemented independently by the server 104 or the terminal 102. For example, the terminal 102 can directly process the passenger flow area image in the target scene, or the server 104 can obtain the passenger flow area image in the target scene from the data storage system and process the passenger flow area image.

[0020] The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.

[0021] In an exemplary embodiment, Figure 2 As shown, a passenger flow counting method based on image generation is provided. The method is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used for explanation, and the steps include the following steps 201 to 207.

[0022] Step 201, obtaining a passenger flow area image in a target scene.

[0023] Step 202: Detect and count complete pedestrians in the passenger flow area image to obtain foreground passenger flow.

[0024] Step 203: remove the complete pedestrians in the passenger flow area image to obtain an occluded image.

[0025] Step 204 , detecting the occluding object in the occluding image, and removing the occluding object in the occluding image to obtain a remaining image area.

[0026] Step 205, input the pixel values ​​in the remaining image area into the image generation model, reconstruct the area where the occluded object is located, and determine the occluded image after the occluded object is reconstructed as the reconstructed image; the image generation model is constructed based on a deep learning model.

[0027] Step 206: Detect and count the complete pedestrians in the reconstructed image to obtain the passenger flow in the blocked area.

[0028] Step 207: determining the final passenger flow in the target scene according to the foreground passenger flow and the blocked area passenger flow.

[0029] The current passenger flow counting method often cannot effectively handle the occlusion problem, resulting in large errors in the statistical results. Implementing the above steps 201 to 207, generating the area where the occluding object is located through image generation technology can effectively solve the occlusion problem and improve the accuracy of passenger flow counting. At the same time, the passenger flow counting method of this embodiment also has the advantages of simple operation and wide application range, and can be widely used in places with large passenger flow such as subways, shopping malls, stations, airports, etc., and has important practical application value.

[0030] In another exemplary embodiment of the present application, in order to reduce the impact of the randomness of the image generation model on the statistical results and further ensure the accuracy of the passenger flow statistics, the above steps 201-207 can be performed multiple times, and the average value of the statistical results of multiple passenger flows is taken as the final passenger flow in the target scene.

[0031] In another exemplary embodiment of the present application, the above step 201 specifically includes: acquiring an original image obtained by photographing the passenger flow area in the target scene; performing image denoising and image enhancement on the original image to obtain an image of the passenger flow area.

[0032] Specifically, a high-definition camera or other equipment deployed in the target scene is used to monitor the passenger flow area to obtain the original image. Since the original image captured by the high-definition camera may be affected by environmental noise, light and other factors, a denoising operation is required. This embodiment uses a mean filter, a median filter or a Gaussian filter method to remove noise from the original image, thereby improving the image quality.

[0033] Mean filtering is a simple linear smoothing filter. When performing mean filtering, the grayscale value of each pixel in the original image is averaged to replace the grayscale value of the pixel. This can reduce the impact of noise on the image and make the image smoother. Mean filtering is suitable for mild Gaussian noise or uniformly distributed noise.

[0034] Median filtering is a nonlinear filtering method that sorts the pixels in the neighborhood around each pixel by grayscale value and then selects the middle value as the new grayscale value of the pixel. Median filtering has a good effect on removing impulse noise because it is not affected by extreme values ​​and can maintain image edge information.

[0035] Gaussian filtering uses a Gaussian function to perform a weighted average of pixels in the neighborhood around a pixel. The closer the pixel is to the center pixel, the greater the weight. The purpose of this is to reduce local high-frequency signals, thereby achieving the effect of removing noise. Gaussian filtering is suitable for Gaussian noise or fuzzy noise.

[0036] In this embodiment, it is necessary to first select a suitable or combined denoising method according to the characteristics of the original image and the specific noise situation (ie, through noise type detection means) to achieve the best denoising effect.

[0037] Among them, the noise type detection means include: pre-denoising evaluation and post-denoising feedback evaluation. Pre-denoising evaluation includes: histogram analysis and local variance analysis. Post-denoising feedback evaluation includes: differential image analysis and filter performance evaluation. Combining the pre-denoising evaluation method with the post-denoising feedback evaluation method, the noise type of the original image collected within a certain period of time can be determined.

[0038] Histogram analysis is to understand the grayscale distribution of an image by analyzing its histogram. If the histogram shows double peaks or discontinuous peaks and valleys, it is affected by impulse noise; and if the histogram is too smooth or skewed, it is caused by Gaussian noise.

[0039] Local variance analysis is to calculate the variance of pixels in the neighborhood around each pixel in the image. If the variance is large, it means that the pixels around the pixel have a large grayscale change and are affected by Gaussian noise; on the contrary, if the variance is small, it is affected by uniformly distributed noise.

[0040] Differential image analysis is to subtract the original image from the filtered image, and then infer the noise type by analyzing the characteristics of the differential image. For example, if extreme values ​​(such as 255 or 0) appear in the differential image, it corresponds to impulse noise; if relatively uniform small values ​​appear in the differential image, it corresponds to Gaussian noise.

[0041] Filter performance evaluation, that is, the noise type can be determined by applying different types of filters (such as mean filter, median filter, Gaussian filter) to the image and observing the effect of the filtered image. Different types of noise will produce different effects after filtering, and the noise type can be inferred by observing the filtered image.

[0042] In order to further improve the clarity and contrast of the image, this embodiment also uses histogram equalization or contrast limited adaptive histogram equalization (CLAHE) and other technologies to enhance the denoised image. The image enhancement operation can enhance the detail information of the image and make the subsequent processing steps more accurate.

[0043] The main idea of ​​histogram equalization is to enhance the contrast of the image by redistributing the grayscale levels of the image so that the histogram of the image is evenly distributed throughout the grayscale range. This method is implemented through the cumulative distribution function, which maps the grayscale value of each pixel in the denoised image to a new grayscale level to achieve the uniformity of the histogram. The specific steps include: calculating the grayscale histogram of the denoised image; calculating the cumulative distribution function; and mapping the original pixel grayscale value to the new grayscale level. This can enhance the overall contrast of the image and make the image clearer.

[0044] Contrast-limited adaptive histogram equalization is an improvement on histogram equalization, which mainly solves the problem of over-enhancement that may be caused when histogram equalization enhances contrast in local areas. CLAHE divides the image into many small local areas and applies histogram equalization in each local area. However, in order to avoid over-enhancement of contrast, CLAHE limits the degree of contrast enhancement in each local area. Specifically, CLAHE calculates the histogram in each local area and controls the amplitude of contrast enhancement by limiting the slope of the cumulative distribution function after histogram equalization. In this way, while enhancing the contrast of the image, it avoids the problems of over-enhancement and noise amplification, thereby improving the quality of the image.

[0045] The process of selecting the noise type corresponding to the above enhancement technology is as follows.

[0046] For Gaussian noise, if the image is slightly disturbed by Gaussian noise, histogram equalization can be used to enhance the image contrast, because histogram equalization can balance the overall contrast, thereby improving image quality. If the image is affected by more serious Gaussian noise, contrast-limited adaptive histogram equalization can be used to enhance the contrast in local areas, while limiting the amplitude of contrast enhancement, which helps to avoid excessive noise enhancement.

[0047] For impulse noise, since impulse noise usually appears as random black and white dots, histogram equalization may make these noise dots more obvious and is not suitable for images interfered by impulse noise. When facing impulse noise, contrast-limited adaptive histogram equalization is often more suitable. Contrast-limited adaptive histogram equalization enhances contrast in local areas and can effectively handle impulse noise and maintain image details.

[0048] In another exemplary embodiment of the present application, step 202 specifically includes: inputting the passenger flow area image into the foreground person detection model to obtain a first bounding box and corresponding person detection information in the passenger flow area image for characterizing the possible existence of foreground persons; the person detection information includes: the center coordinates of the first bounding box, the width and height of the first bounding box, the confidence that the foreground person exists in the first bounding box, and the category information of whether it is a person; the foreground person detection model is constructed based on a deep learning model. According to the person detection information and the pixels in the first bounding box, determine the bounding box in the first bounding box that contains the key points of the complete human body. Count the number of bounding boxes that contain the key points of the complete human body to obtain the foreground passenger flow. The deep learning model of this embodiment can use the Faster R-CNN model or the YOLO model.

[0049] In another exemplary embodiment of the present application, step 203 specifically includes: using semantic segmentation technology to process the area in the bounding box containing the key points of the complete human body to determine the outline of the complete pedestrian; removing the complete pedestrian in the passenger flow area image according to the outline of the complete pedestrian to obtain an occluded image.

[0050] In another exemplary embodiment of the present application, step 204 specifically includes: inputting the occluded image into the occluded object detection model to obtain a second bounding box and corresponding object detection information in the occluded image for characterizing the possible existence of the occluded object; the object detection information includes: the center coordinates of the second bounding box, the width and height of the second bounding box, the confidence that the occluded object exists in the second bounding box, and whether it is the category information of the object; the occluded object detection model is built based on a deep learning model. Determine the bounding box containing the occluded object in the second bounding box according to the object detection information. Use semantic segmentation technology to process the area in the bounding box containing the occluded object to determine the outline of the occluded object. Remove the occluded object in the occluded image according to the outline of the occluded object to obtain an occluded image. The deep learning model of this embodiment can use a Faster R-CNN model or a YOLO model.

[0051] In another exemplary embodiment of the present application, the method for determining the image generation model specifically includes: obtaining a passenger flow area training image; the passenger flow area training image does not contain an occluding object. Partially occluding the passenger flow area training image with an occluding object to obtain a passenger flow area marked image, and removing the partially occluded area in the passenger flow area marked image to obtain a remaining training area. Construct a first training set based on the pixel values ​​of the remaining training area and the pixel values ​​of the partially occluded area in the passenger flow area training image. Input the first training set into a variational autoencoder (VAE), train the variational autoencoder with the goal of minimizing the sum of reconstruction loss and KL divergence loss, and determine the trained variational autoencoder as the image generation model.

[0052] In another exemplary embodiment of the present application, before step 206, the passenger flow counting method based on image generation also includes: calculating the difference between the reconstructed image and the occluded image before reconstruction of the occluded object; when the difference is greater than a set threshold, detecting and counting the complete pedestrians in the reconstructed image to obtain the passenger flow in the occluded area; when the difference is less than or equal to the set threshold, regenerating the reconstructed image until the difference is greater than the set threshold or reaches the set number of iterations, detecting and counting the complete pedestrians in the reconstructed image to obtain the passenger flow in the occluded area.

[0053] The following takes the subway as an example of the target scene to further introduce the above-mentioned passenger flow statistics method based on image generation.

[0054] Step 1: Acquisition of video stream of passenger flow area and denoising and enhancement of images of passenger flow area.

[0055] 1) Collect the video stream of the passenger flow area and obtain the real-time passenger flow area image. In this embodiment, high-definition cameras and other equipment are deployed in the subway entrance / exit, subway platform, station hall and other areas to monitor the passenger flow area, obtain the real-time video stream, and perform the subsequent steps for each frame of the passenger flow area image in the real-time video stream to realize passenger flow statistics. In actual applications, the real-time video stream is sampled every 10 frames.

[0056] 2) Image denoising: Since the images captured by the high-definition camera may be affected by environmental noise, light and other factors, denoising is required. This embodiment uses mean filtering, median filtering or Gaussian filtering to remove noise from the image, thereby improving image quality.

[0057] 3) Image enhancement: In order to further improve the clarity and contrast of the image, this embodiment adopts technologies such as histogram equalization and contrast-limited adaptive histogram equalization to enhance the image.

[0058] Step 2: Identification of the area where the occluded object is located.

[0059] In this step, first, foreground person detection is performed, then foreground object detection is performed, and finally image reconstruction and re-identification techniques are used to improve accuracy.

[0060] Sub-step 1: Foreground person detection.

[0061] Foreground person detection is achieved by selecting a deep learning model. The model will output a series of bounding boxes to indicate the positions of the detected people. Then, the posture of the detected people is estimated to ensure that the complete pedestrian is recognized. This embodiment specifically selects the YOLO model for implementation.

[0062] The YOLO model is a deep learning model for real-time object detection. The model takes the entire image as input and uses a convolutional neural network (CNN) to extract the features of each bounding box in the image. In order to identify the category of a person, the YOLO model adds an additional convolutional layer in the middle or bottom layer of the network to capture the features of the person's posture. Its output is a series of bounding boxes and their corresponding detection information {(a, b, w, h, c, class)}, where a and b represent the center coordinates of the bounding box, w and h represent the width and height of the bounding box, c represents the confidence, and class represents the category. Only when the complete human key points are detected, it is considered to be a complete pedestrian and counted. The human key points include: 21 main key points, including the top of the head, facial features, neck, limbs and other major joints.

[0063] The YOLO model can predict the category probability to which each bounding box belongs. These category probabilities represent the possibility that the object belongs to the human body and are usually normalized using the softmax function.

[0064] For confidence, the confidence score is usually a value between 0 and 1, where: a score close to 1 means that the model is very confident that the bounding box contains human parts or behaviors. A score close to 0 means that the model believes that the bounding box does not contain human parts, or the model is not very confident about its detection results.

[0065] Therefore, in this embodiment, a confidence score threshold is set, and if the confidence score threshold is exceeded, it is determined that a person is included in the bounding box.

[0066] A specific implementation process of the YOLO model can be described as follows.

[0067] S11, divide the input image into grids: First, divide the input image into a grid of a fixed size, usually a grid of size S×S.

[0068] S12, predict bounding boxes in each grid: In each grid cell, predict a fixed number (usually B) of bounding boxes.

[0069] S13, predicted category probability: Each grid cell predicts the probability distribution class of C categories, which are the target categories to be detected by the model, that is, human parts.

[0070] S14, calculate the confidence of the predicted box: associate the confidence of each bounding box with the probability of containing the object and the position of the bounding box.

[0071] S15, apply non-maximum suppression (NMS): In order to eliminate overlapping bounding boxes, non-maximum suppression is used to select the bounding box with the highest confidence and remove other bounding boxes with an IoU (intersection over union) higher than a threshold.

[0072] S16, counting.

[0073] Sub-step 2: occluded object detection.

[0074] The detection of occluded objects is similar to that of foreground people, and will not be described in detail here.

[0075] Sub-step three: removal of occluding objects.

[0076] First, semantic segmentation technology is used to divide the areas of different objects in the image. Secondly, people and occluded objects are matched with each other. Finally, the occluded objects are removed from the image.

[0077] The specific process of removing occluded objects can be described as follows.

[0078] S21, Image Segmentation: First, use semantic segmentation technology to segment the image, mark the regions of different objects, and obtain the semantic category label of each pixel and the segmentation mask indicating which object each pixel belongs to. The semantic category label is the object category defined by COCO.

[0079] S22, occluded object detection: According to the segmentation result of step S21, the area where the occluded object is located is identified, and the occluded object is detected by analyzing the segmentation mask.

[0080] S23, occluded object removal: Once the occluded object is detected, it can be removed from the image.

[0081] Step 3: Image reconstruction and re-identification.

[0082] The pixel values ​​in the image are continuous between the occluded area and the non-occluded area. The occluded area is estimated by using the existing information in the image. This embodiment constructs an image occluded area reconstruction method based on the VAE model.

[0083] Sub-step 1: VAE model training and learning.

[0084] First, the goal of the VAE model is to learn a latent space, map the input image x to the distribution q(z|x) in the latent space through the encoder, and then reconstruct the complete image x' from the sample z in the latent space through the decoder.

[0085] The loss function of the VAE model usually consists of two parts: reconstruction loss and KL divergence loss.

[0086] The reconstruction loss is used to measure the difference between the reconstructed image and the original image, and its calculation formula is as follows.

[0087] .

[0088] KL divergence loss is used to measure the difference between the latent space distribution and the standard normal distribution to ensure the smoothness of the latent space. The calculation formula of KL divergence loss is as follows.

[0089] .

[0090] in, represents reconstruction losses; represents the expectation of the distribution q(z|x); P(x|z) is the probability of restoring the latent space z to the image x; represents the KL divergence loss; p(z) is the normal distribution of the latent space z; Indicates KL divergence calculation; represents the probability of generating z from image x.

[0091] For passenger flow area images with the subway as the target scene, some areas are randomly blocked, and then the VAE model is trained with the goal of reconstructing the area to obtain an image generation model.

[0092] Sub-step 2: Restoration of occluded areas.

[0093] Based on the above image generation model, the pixel values ​​around the occluded area are used as input, the pixel values ​​of the occluded area are predicted, and the occluded area is restored to obtain a reconstructed image. If the difference between the reconstructed image and the occluded image before reconstruction of the occluded object is greater than the set threshold, the complete pedestrians in the reconstructed image can be detected and counted to obtain the passenger flow in the occluded area; when the difference is less than or equal to the set threshold, the reconstructed image is regenerated until the difference is greater than the set threshold or the set number of iterations is reached, and the complete pedestrians in the reconstructed image are detected and counted to obtain the passenger flow in the occluded area.

[0094] The final passenger flow in the subway passenger flow area is obtained based on the foreground passenger flow obtained by counting in sub-step 1 and the passenger flow in the blocked area obtained in this step.

[0095] Step 4: Iterative optimization.

[0096] In this step, in order to ensure the accuracy of the statistical results, the steps of occluded area recognition and image reconstruction are repeated, and the statistical results of multiple passenger flows are averaged as the final passenger flow of the subway passenger flow area.

[0097] Through the above steps, subway passenger flow statistics based on image generation technology are realized, the accuracy of subway passenger flow statistics is improved, and reliable data support is provided for the statistics and analysis of subway passenger flow.

[0098] Based on the same inventive concept, the embodiment of the present application also provides a passenger flow counting device based on image generation for implementing the passenger flow counting method based on image generation involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more embodiments of the passenger flow counting device based on image generation provided below can refer to the limitations of the passenger flow counting method based on image generation above, and will not be repeated here.

[0099] In an exemplary embodiment, Figure 3 As shown, a passenger flow counting device based on image generation is provided, including: an image acquisition module 301, a foreground counting module 302, a pedestrian removal module 303, an object detection and removal module 304, an image reconstruction module 305, an occluded area counting module 306 and a passenger flow determination module 307.

[0100] The image acquisition module 301 is used to acquire the passenger flow area image in the target scene. The foreground counting module 302 is used to detect and count the complete pedestrians in the passenger flow area image to obtain the foreground passenger flow. The pedestrian removal module 303 is used to remove the complete pedestrians in the passenger flow area image to obtain the occluded image. The object detection and removal module 304 is used to detect the occluded objects in the occluded image and remove the occluded objects in the occluded image to obtain the remaining image area. The image reconstruction module 305 is used to input the pixel values ​​in the remaining image area into the image generation model, reconstruct the area where the occluded object is located, and determine the occluded image after the occluded object is reconstructed as the reconstructed image; the image generation model is constructed based on the deep learning model. The occluded area counting module 306 is used to detect and count the complete pedestrians in the reconstructed image to obtain the passenger flow in the occluded area. The passenger flow determination module 307 is used to determine the final passenger flow in the target scene according to the foreground passenger flow and the occluded area passenger flow.

[0101] The passenger flow counting device based on image generation in this embodiment generates the area where the occluded object is located through image generation technology, and then detects the passenger flow in the occluded area, and determines the final passenger flow by the foreground passenger flow and the occluded area passenger flow, thereby improving the accuracy of passenger flow statistics.

[0102] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store passenger flow area images in the target scene. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a passenger flow statistics method based on image generation is implemented.

[0103] Those skilled in the art will understand that Figure 4The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0104] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0105] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0106] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0107] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0108] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, etc., but is not limited thereto.

[0109] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0110] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A passenger flow counting method based on image generation, characterized in that: The passenger flow counting method based on image generation includes: Acquire the passenger flow area image in the target scene; Detecting and counting complete pedestrians in the passenger flow area image to obtain foreground passenger flow; Removing complete pedestrians from the passenger flow area image to obtain an occluded image; Detecting the occluded object in the occluded image and removing the occluded object in the occluded image to obtain a remaining image area; Inputting pixel values ​​in the remaining image area into an image generation model, reconstructing the area where the occluded object is located, and determining the occluded image after the occluded object is reconstructed as the reconstructed image; the image generation model is constructed based on a deep learning model; Detecting and counting complete pedestrians in the reconstructed image to obtain the passenger flow in the blocked area; Determining a final passenger flow in the target scene according to the foreground passenger flow and the blocked area passenger flow; The method for determining the image generation model specifically includes: Acquire a passenger flow area training image; the passenger flow area training image does not contain any occluding objects; Using an occluding object to partially occlude the passenger flow area training image to obtain a passenger flow area marked image, and removing the partially occluded area in the passenger flow area marked image to obtain a remaining training area; Constructing a first training set according to the pixel values ​​of the remaining training area and the pixel values ​​of the partially blocked area in the passenger flow area training image; The first training set is input into a variational autoencoder, the variational autoencoder is trained with the goal of minimizing the sum of reconstruction loss and KL divergence loss, and the trained variational autoencoder is determined as the image generation model.

2. The passenger flow counting method based on image generation according to claim 1, characterized in that: Before detecting and counting complete pedestrians in the reconstructed image to obtain the passenger flow in the blocked area, the passenger flow counting method based on image generation further includes: Calculating the difference between the reconstructed image and the occluded image before reconstruction of the occluded object; When the difference is greater than a set threshold, the complete pedestrians in the reconstructed image are detected and counted to obtain the passenger flow in the blocked area.

3. The passenger flow counting method based on image generation according to claim 1, characterized in that: Detecting and counting complete pedestrians in the passenger flow area image to obtain foreground passenger flow, specifically includes: Input the passenger flow area image into a foreground person detection model to obtain a first bounding box for representing possible foreground persons in the passenger flow area image and corresponding person detection information; the person detection information includes: the center coordinates of the first bounding box, the width and height of the first bounding box, the confidence that the foreground person exists in the first bounding box, and the category information of whether the foreground person is a person; the foreground person detection model is constructed based on a deep learning model; Determine, according to the person detection information and the pixels in the first bounding box, a bounding box containing key points of a complete human body in the first bounding box; The number of bounding boxes containing complete human key points is counted to obtain the foreground passenger flow.

4. The passenger flow counting method based on image generation according to claim 3 is characterized in that: The complete pedestrians in the passenger flow area image are removed to obtain an occluded image, which specifically includes: Semantic segmentation technology is used to process the area in the bounding box containing the key points of the complete human body to determine the outline of the complete pedestrian; The complete pedestrians in the passenger flow area image are removed according to the contours of the complete pedestrians to obtain an occluded image.

5. The passenger flow counting method based on image generation according to claim 1, characterized in that: Detecting the occluded object in the occluded image and removing the occluded object in the occluded image to obtain a remaining image area specifically includes: Input the occluded image into an occluded object detection model to obtain a second bounding box and corresponding object detection information in the occluded image for characterizing possible occluded objects; the object detection information includes: the center coordinates of the second bounding box, the width and height of the second bounding box, the confidence that the occluded object exists in the second bounding box, and whether it is an object category information; the occluded object detection model is constructed based on a deep learning model; Determine, according to the object detection information, a bounding box in the second bounding box that contains the occluded object; Semantic segmentation technology is used to process the area in the bounding box containing the occluded object to determine the outline of the occluded object; The occluding object in the occluding image is removed according to the contour of the occluding object to obtain the occluding image.

6. The passenger flow counting method based on image generation according to claim 1, characterized in that: Obtain the passenger flow area image in the target scene, including: Acquire an original image obtained by photographing a passenger flow area in a target scene; The original image is subjected to image denoising and image enhancement to obtain a passenger flow area image.

7. A passenger flow counting device based on image generation, characterized in that: The passenger flow counting device based on image generation comprises: An image acquisition module, used to acquire an image of a passenger flow area in a target scene; A foreground counting module is used to detect and count complete pedestrians in the passenger flow area image to obtain the foreground passenger flow; A pedestrian removal module, used to remove complete pedestrians from the passenger flow area image to obtain an occluded image; An object detection and removal module, used to detect the occluding object in the occluding image and remove the occluding object in the occluding image to obtain a remaining image area; An image reconstruction module, used for inputting the pixel values ​​in the remaining image area into an image generation model, reconstructing the area where the occluding object is located, and determining the occluding image after the occluding object is reconstructed as the reconstructed image; the image generation model is constructed based on a deep learning model; An occluded area counting module is used to detect and count complete pedestrians in the reconstructed image to obtain the passenger flow in the occluded area; A passenger flow determination module, used to determine the final passenger flow in the target scene according to the foreground passenger flow and the blocked area passenger flow; The method for determining the image generation model specifically includes: Acquire a passenger flow area training image; the passenger flow area training image does not contain any occluding objects; Using an occluding object to partially occlude the passenger flow area training image to obtain a passenger flow area marked image, and removing the partially occluded area in the passenger flow area marked image to obtain a remaining training area; Constructing a first training set according to the pixel values ​​of the remaining training area and the pixel values ​​of the partially blocked area in the passenger flow area training image; The first training set is input into a variational autoencoder, the variational autoencoder is trained with the goal of minimizing the sum of reconstruction loss and KL divergence loss, and the trained variational autoencoder is determined as the image generation model.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the passenger flow counting method based on image generation according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the passenger flow counting method based on image generation described in any one of claims 1 to 6 is implemented.

Images