Data acquisition device for crowd statistics based on AI technology

Abstract

The invention relates to the technical field of computer vision, in particular to an AI technology-based data acquisition device for crowd statistics, which comprises an acquisition module for acquiring crowd density image data and real-time video image data in various scenes; the auxiliary acquisition module is used for acquiring interactive crowd density image data; the data preprocessing module is used for preprocessing the image data; the model training module is used for constructing and training a crowd statistical model; the model optimization module is used for compressing and optimizing the trained crowd statistical model; the crowd quantity statistics module is used for generating a crowd quantity by utilizing the optimized crowd statistics model; and the mobile client module is used for providing a mobile APP client for the user. According to the method, video image data containing diversified scenes and crowd density are collected, and deep learning and model optimization are carried out by using a mercuric chloride AI calculation framework, so that the detection precision and robustness of the crowd density in a complex scene can be improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a data collection device for crowd statistics based on AI technology. Background Art

[0002] Ascend AI is an artificial intelligence computing framework independently developed by Huawei. Its efficient computing power and flexible software ecosystem provide strong technical support for dense crowd counting. By collecting a large amount of video image data covering different scenes and crowds of different densities, and using the Ascend AI computing framework for model training and tuning, we have developed deep learning models that can accurately identify and count people. These models have been widely used in security monitoring, public transportation and other fields, providing strong support for urban management, public safety, etc.

[0003] Existing technologies use GPU acceleration technology to speed up the training and reasoning process of deep learning models, and use edge computing technology to deploy crowd counting models on terminal devices to achieve efficient crowd counting. Although AI models have made significant progress in dense crowd counting, in some complex scenarios, such as when the crowd density is extremely high, the counting accuracy of the model still needs to be improved.

[0004] In summary, how to solve the problem of low counting accuracy of the model under conditions of extremely high crowd density has become a difficult problem that needs to be solved urgently in this field. Therefore, it is necessary to propose a data collection device for crowd statistics based on AI technology. Summary of the invention

[0005] To solve the above problems, the present invention provides a data acquisition device for crowd statistics based on AI technology. By combining Ascend AI technology with target detection algorithm and semantic segmentation algorithm, the data acquisition device of the present invention can adapt to and optimize the crowd statistics needs in different scenarios and improve the detection accuracy of different scenarios. By collecting video image data containing diverse scenes and crowd density, and using the Ascend AI computing framework for deep learning and model tuning, the data acquisition device of the present invention can improve the detection accuracy and robustness of crowd density in complex scenarios.

[0006] In order to achieve the above-mentioned purpose, the technical solution of the present invention is as follows: a data acquisition device for crowd statistics based on AI technology, including an acquisition module, an auxiliary acquisition module, a data preprocessing module, a model training module, a model optimization module, a crowd quantity statistics module and a mobile client module.

[0007] The acquisition module is used to collect crowd density image data and real-time video image data in various scenes using a camera.

[0008] The auxiliary acquisition module is used to set the camera on the large screen, play the interactive information through the large screen, and the camera collects the number of people interacting with the interactive information in the scene in real time to obtain the interactive crowd density image data.

[0009] The data preprocessing module is used to perform preprocessing operations on the crowd density image data and the interactive crowd density image data to obtain the preprocessed crowd density image data and the interactive crowd density image data, and at the same time, the preprocessed crowd density image data and the interactive crowd density image data are annotated to mark the position and number of the crowd to obtain the annotated image data.

[0010] The model training module is used to build an algorithm framework based on a convolutional neural network and combined with a semantic segmentation algorithm, and run the algorithm framework on the Ascend AI platform. At the same time, it uses the underlying heterogeneous computing capabilities and compilation optimization technology of the Ascend AI computing framework in the Ascend AI platform to input the labeled image data into the algorithm framework to train the crowd statistics model and obtain the trained crowd statistics model.

[0011] The model optimization module is used to utilize the operator optimization and hardware acceleration functions provided by the Ascend AI computing framework in the Ascend AI platform, and to compress and optimize the trained crowd statistics model using pruning and quantization technologies to obtain an optimized crowd statistics model.

[0012] The crowd counting module is used to analyze the real-time video image data using the optimized crowd counting model to obtain the real-time crowd count information.

[0013] The mobile client module is used to provide users with a mobile APP client. The mobile APP client includes a user interaction interface. The user interaction interface is used to display the crowd quantity information, queue management information and heat map of the crowd in the scene in real time in combination with the scene map.

[0014] Furthermore, the cameras in the acquisition module collect data from a variety of scenes including parks, playgrounds, shopping malls, school cafeterias, and stations.

[0015] Furthermore, the interactive information in the auxiliary acquisition module includes one or more of video interactive information, image interactive information, advertisement interactive information and QR code interactive information.

[0016] Furthermore, the preprocessing operation in the data preprocessing module includes the following steps:

[0017] Step 1: Data cleaning: remove redundant, repeated and invalid parts of the crowd density image data and interactive crowd density image data to obtain cleaned crowd density image data and interactive crowd density image data.

[0018] Step 2: Data denoising: denoising the cleaned crowd density image data and interactive crowd density image data using filtering technology to obtain denoised crowd density image data and interactive crowd density image data.

[0019] Step three, data enhancement: rotate, scale and flip the denoised crowd density image data and interactive crowd density image data to obtain pre-processed crowd density image data and interactive crowd density image data.

[0020] Furthermore, the model training module includes an accelerated training unit, which is used to use a GPU and an Ascend AI processor to accelerate the training of the crowd statistics model.

[0021] Furthermore, the semantic segmentation algorithm in the model training module includes one or more of the DeepLab algorithm and the Mask R-CNN algorithm.

[0022] Furthermore, the model optimization module includes a model evaluation unit, which is used to evaluate the recognition accuracy and counting error rate of the crowd statistics model on the validation set and generate an evaluation result.

[0023] Furthermore, the model optimization module also includes a model optimization unit, which is used to adjust the network structure and learning rate of the convolutional neural network according to the evaluation results, and use the SGD algorithm to adjust the operating parameters and structure of the optimized crowd statistics model in real time.

[0024] Furthermore, the crowd counting module includes an edge computing unit, which is used to analyze real-time video image data using an optimized crowd counting model deployed on a computing device near the large screen to obtain preliminary crowd count information.

[0025] Furthermore, the crowd counting module also includes an end-cloud collaboration unit, which uses the data synchronization and transmission technology in the Ascend AI platform to upload preliminary crowd counting information to the cloud. The cloud then sends the preliminary crowd counting information to the model training module to assist in the training of the crowd statistics model.

[0026] The technical principles of the above scheme are as follows:

[0027] First, the acquisition module collects a large amount of video image data containing different scenes to ensure the diversity and representativeness of the data in order to train a model with generalization ability; then, the data preprocessing module preprocesses the collected video image data to improve data quality and reduce the difficulty of model training. At the same time, the preprocessed image data is annotated to mark the location and number of people and obtain the annotation results. The annotation results will be used as supervision information for model training to guide the model to learn how to identify and count people; then, the model training module integrates the designed crowd statistics model into the Ascend AI platform, uses the powerful computing power provided by the framework to train the model, uses the annotated image data to train the model, and updates the model parameters through the back propagation algorithm so that the model gradually learns the ability to identify and count people. The model optimization module tunes the model performance according to the performance of the model on the validation set; finally, the crowd quantity statistics module sends the real-time crowd quantity information to the mobile client module to display the crowd quantity to users in the mobile APP client.

[0028] The present invention adopts the above scheme to have the following beneficial effects:

[0029] 1. The present invention combines Ascend AI technology with target detection algorithm and semantic segmentation algorithm to provide an efficient and accurate data collection device for crowd counting, which can adapt to different scenarios and crowd densities. By preprocessing and annotating image data, a crowd counting model with high accuracy and robustness is trained, which effectively solves the problem of low model counting accuracy in complex scenarios.

[0030] 2. The data acquisition device of the present invention utilizes the powerful computing power of the Ascend AI platform to achieve rapid training and optimization of the model; through accelerated training of the GPU and Ascend AI processor, as well as operator optimization and hardware acceleration functions, the training efficiency and performance of the model are significantly improved; at the same time, pruning technology and quantization technology are used to compress and optimize the model, reducing the complexity and computational complexity of the model, making it more suitable for deployment and use in practical applications.

[0031] 3. The present invention can deploy the trained model in various queue management systems; for example, in the queue management system of an amusement park, tourists can view the real-time flow of people through their mobile phones, understand the queue time of each amusement facility, avoid long waiting in line, and improve the tourists' visiting experience; at the same time, intensive population statistics can help the amusement park management to provide crowd flow reports, so that the amusement park can adjust its operation strategy in time.

[0032] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 Schematic diagram of the structure of a data collection device for crowd statistics based on AI technology in an embodiment of the present invention.

[0034] Figure 2 Schematic diagram of the process of extracting features using a convolutional neural network in a crowd statistics model in an embodiment of the present invention.

[0035] Figure 3 Schematic diagram of the structure of the crowd statistics model in the embodiment of the present invention.

[0036] Figure 4 Schematic diagram of the statistical effect of the crowd counting module in an embodiment of the present invention.

[0037] Figure 5 Schematic diagram of a visualization interface in an embodiment of the present invention.

[0038] Figure 6 Schematic diagram of queue management information of mobile APP in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The following is further described in detail through specific implementation methods:

[0040] As attached Figure 1-Figure 6 As shown: A data collection device for crowd statistics based on AI technology, including a collection module, an auxiliary collection module, a data preprocessing module, a model training module, a model optimization module, a crowd quantity statistics module and a mobile client module.

[0041] The acquisition module is used to collect crowd density image data and real-time video image data in various scenes using a camera.

[0042] Among them, the camera in the acquisition module collects multiple scenes including parks, playgrounds, shopping malls, school cafeterias and stations; the scene used in this embodiment is the playground.

[0043] The auxiliary acquisition module is used to set the camera on the large screen, play the interactive information through the large screen, and the camera collects the number of people interacting with the interactive information in the scene in real time to obtain the interactive crowd density image data.

[0044] The interactive information in the auxiliary acquisition module includes one or more of video interactive information, image interactive information, advertisement interactive information and QR code interactive information. In this embodiment, advertisement interactive information is used as the interactive information.

[0045] For example, high-definition cameras are deployed in various key areas of the amusement park, such as entrances, exits, around amusement facilities and dining areas. The cameras are used to collect crowd density image data and real-time video image data. At this time, by using a large screen to play interactive advertising information, tourists in the amusement park can be attracted to look up and watch the interactive advertising information on the large screen. At this time, the camera set next to the large screen can clearly capture the facial contours of the interacting tourists, making the crowd density image data and real-time video image data collected by the acquisition module clearer and more accurate, and at the same time obtaining the number of people interacting with the interactive information.

[0046] The data preprocessing module is used to perform preprocessing operations on the crowd density image data and the interactive crowd density image data to obtain the preprocessed crowd density image data and the interactive crowd density image data, and at the same time, the preprocessed crowd density image data and the interactive crowd density image data are annotated to mark the position and number of the crowd to obtain the annotated image data.

[0047] The preprocessing operation in the data preprocessing module includes the following steps:

[0048] Step 1: Data cleaning: remove redundant, repeated and invalid parts of the crowd density image data and interactive crowd density image data to obtain cleaned crowd density image data and interactive crowd density image data.

[0049] Step 2: Data denoising: denoising the cleaned crowd density image data and interactive crowd density image data using filtering technology to obtain denoised crowd density image data and interactive crowd density image data.

[0050] Step three, data enhancement: rotate, scale, flip and other operations are performed on the denoised crowd density image data and interactive crowd density image data to obtain pre-processed crowd density image data and interactive crowd density image data.

[0051] For example, during the data cleaning stage, the system will first identify and remove those crowd density image data and interactive crowd density image data that are blurred or overlapped due to camera obstruction, dark or bright light, and movement of tourists. These redundant, repeated or invalid data parts will not only increase the complexity of subsequent processing, but may also have a negative impact on the final crowd statistics results. Through data cleaning, the system can ensure that the data processed subsequently is clear, accurate and valid.

[0052] The amusement park is an environment full of dynamic changes. The movement of tourists, the operation of amusement facilities and changes in the surrounding environment may cause noise interference to the image data captured by the camera. The data preprocessing module uses Gaussian filtering, mean filtering or median filtering to denoise the cleaned crowd density image data and interactive crowd density image data. It can effectively smooth the image, reduce the impact of noise on image quality, and make the image clearer and more accurate.

[0053] In order to improve the generalization and robustness of the crowd statistics model, the data preprocessing module will perform a series of data enhancement operations on the denoised crowd density image data and interactive crowd density image data, including image rotation, scaling and flipping. Through data enhancement, the system can generate more diverse training data to help the model better learn the crowd characteristics in different scenarios, thereby improving the model's recognition accuracy and counting precision.

[0054] The data preprocessing module will flexibly adjust the parameters and strategies of data cleaning, denoising and data enhancement according to the actual situation of the amusement park and the characteristics of the image data captured by the camera; for example, in crowded areas such as the entrance and exit of the amusement park, the system may increase the intensity of data enhancement to better cope with challenges such as crowds and occlusion; while in relatively open areas such as around amusement facilities and dining areas, the data processing module will reduce the intensity of data enhancement to avoid image distortion caused by excessive enhancement.

[0055] After preprocessing by the data preprocessing module, the data preprocessing module generates a series of clear, accurate and diverse labeled image data, which will be used in the subsequent crowd statistics model training and optimization process.

[0056] Through the preprocessing operation of the data preprocessing module, it can ensure that the collected crowd density image data and interactive crowd density image data are of higher quality and more valuable, thereby providing more accurate and reliable data support for crowd counting and management of amusement parks.

[0057] The model training module is used to build an algorithm framework based on a convolutional neural network and combined with a semantic segmentation algorithm, and run the algorithm framework on the Ascend AI platform. At the same time, it uses the underlying heterogeneous computing capabilities and compilation optimization technology of the Ascend AI computing framework in the Ascend AI platform to input the labeled image data into the algorithm framework to train the crowd statistics model and obtain the trained crowd statistics model.

[0058] The model optimization module is used to utilize the operator optimization and hardware acceleration functions provided by the Ascend AI computing framework in the Ascend AI platform, and to compress and optimize the trained crowd statistics model using pruning and quantization technologies to obtain an optimized crowd statistics model.

[0059] Among them, the model training module includes an accelerated training unit, which is used to use GPU and Ascend AI processor to accelerate the training of the crowd statistics model; the semantic segmentation algorithm in the model training module includes one or more of the DeepLab algorithm and the Mask R-CNN algorithm; in this embodiment, the DeepLab algorithm is selected as the semantic segmentation algorithm; the model optimization module includes a model evaluation unit, which is used to evaluate the recognition accuracy and counting error rate of the crowd statistics model on the verification set to generate an evaluation result.

[0060] The model optimization module also includes a model optimization unit, which is used to adjust the network structure and learning rate of the convolutional neural network according to the evaluation results, and use the SGD algorithm to adjust the operating parameters and structure of the optimized crowd statistics model in real time.

[0061] For example, in the model training module, the algorithm framework built based on convolutional neural network and DeepLab semantic segmentation algorithm can efficiently identify and segment crowd areas in images; the DeepLab algorithm effectively improves the accuracy and robustness of crowd segmentation by introducing void convolution and conditional random fields.

[0062] The Ascend AI platform plays a vital role when inputting labeled image data into the algorithm framework for training; the Ascend AI platform can efficiently process large-scale image data and accelerate the model training process; at the same time, the compilation optimization technology provided by the Ascend AI computing framework further improves training efficiency and shortens model training time.

[0063] In the model optimization module, the Ascend AI platform provides operator optimization and hardware acceleration functions; by pruning and quantizing the trained crowd statistics model, the complexity and computational complexity of the crowd statistics model can be reduced, and the operating efficiency of the crowd statistics model can be improved. Pruning technology simplifies the structure of the crowd statistics model by removing unimportant weights and connections in the model; while quantization technology converts the weights and activation values ​​of the crowd statistics model from floating point numbers to low-precision integers, further reducing the storage requirements and computational complexity of the crowd statistics model; at the same time, the accelerated training unit can increase the speed of crowd statistics model training; the parallel computing capabilities of the GPU and Ascend AI processor enable the crowd statistics model to be trained in a shorter time.

[0064] In the model optimization module, the model evaluation unit uses the validation set data to comprehensively evaluate the trained crowd statistics model, focusing on the recognition accuracy and counting error rate of the model, and generates a detailed evaluation report; the model optimization unit adjusts the network structure of the convolutional neural network according to the evaluation results, such as increasing or decreasing the number of convolution layers and adjusting the convolution kernel size, in order to find the best network configuration and improve the recognition ability and generalization performance of the model; on the other hand, by setting appropriate learning rates at different stages, it can ensure that the crowd statistics model can converge quickly and avoid overfitting during the training process.

[0065] The SGD algorithm of the model optimization unit is used to adjust the operating parameters and structure of the optimized crowd statistics model in real time; the SGD algorithm randomly selects a small batch of data for gradient update, which not only speeds up the training speed, but also helps the crowd statistics model to escape the local optimal solution, further improving the performance of the crowd statistics model, thereby enabling the crowd statistics model to accurately identify and count the number of people in the amusement park, providing strong data support for the management and operation of the amusement park.

[0066] The crowd counting module is used to analyze the real-time video image data using the optimized crowd counting model to obtain the real-time crowd count information.

[0067] The mobile client module is used to provide users with a mobile APP client. The mobile APP client includes a user interaction interface. The user interaction interface is used to display the crowd quantity information, queue management information and heat map of the crowd in the scene in real time in combination with the scene map.

[0068] Among them, the crowd counting module includes an edge computing unit, which is used to analyze real-time video image data using an optimized crowd counting model deployed on a computing device near the large screen to obtain preliminary crowd count information.

[0069] The crowd counting module also includes an end-cloud collaboration unit, which uses the data synchronization and transmission technology in the Ascend AI platform to upload preliminary crowd counting information to the cloud. The cloud then sends the preliminary crowd counting information to the model training module to assist in the training of the crowd statistics model.

[0070] For example, when visitors enter the amusement park, cameras deployed in various key areas will capture video images in real time, and the edge computing unit in the crowd counting module will use the optimized crowd counting model to quickly analyze these video images.

[0071] The edge computing unit can quickly identify and count the number of people in the video image, and then upload the preliminary crowd number information to the cloud through the end-cloud collaboration unit. After receiving this information, the cloud will further process and analyze it to generate a more detailed report on the distribution and flow of people; at the same time, the cloud will also send this preliminary crowd number information to the model training module as a supplement to the training data, helping the model to continuously optimize and update, and improve the accuracy and efficiency of recognition.

[0072] In the amusement park queue management system, the crowd statistics module can monitor the queue situation in front of each amusement facility in real time, automatically adjust the width and number of queue channels according to the number of people, and optimize the queuing experience of tourists; the mobile APP client can also combine the scene map to display the crowd number information and queue management information of each area in real time on the mobile APP client, so that tourists can understand the crowd level and queue situation of the amusement park at any time and make more reasonable play plans.

[0073] At the same time, the mobile APP client also provides a crowd heat map function, which indicates the crowd density in different areas through the depth of color. Dark colors represent high crowd density, and light colors represent low crowd density. This helps tourists intuitively understand the overall crowd distribution of the amusement park. It not only improves the tourists' entertainment experience, but also provides important data support for the management and operation of the amusement park.

[0074] Through the collaborative work of the crowd counting module and the mobile client module, the amusement park can accurately monitor and manage the flow of tourists, optimize the tourists' play experience, and improve the operational efficiency of the amusement park.

[0075] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. A data collection device for crowd statistics based on AI technology, characterized in that: It includes collection module, auxiliary collection module, data preprocessing module, model training module, model optimization module, crowd counting module and mobile client module; A collection module, used to collect crowd density image data and real-time video image data in various scenes using a camera; The auxiliary acquisition module is used to set the camera on the large screen, play the interactive information through the large screen, and the camera collects the number of people interacting with the interactive information in the scene in real time to obtain the interactive crowd density image data; A data preprocessing module is used to perform preprocessing operations on the crowd density image data and the interactive crowd density image data to obtain the preprocessed crowd density image data and the interactive crowd density image data, and to mark the preprocessed crowd density image data and the interactive crowd density image data to mark the position and number of the crowd to obtain the marked image data; The model training module is used to build an algorithm framework based on a convolutional neural network and combined with a semantic segmentation algorithm, and run the algorithm framework on the Ascend AI platform. At the same time, the underlying heterogeneous computing capabilities and compilation optimization technology of the Ascend AI computing framework in the Ascend AI platform are used to input the labeled image data into the algorithm framework to train the crowd statistics model and obtain the trained crowd statistics model. The model optimization module is used to utilize the operator optimization and hardware acceleration functions provided by the Ascend AI computing framework in the Ascend AI platform, and to compress and optimize the trained crowd statistics model using pruning and quantization technologies to obtain an optimized crowd statistics model; The crowd counting module is used to analyze the real-time video image data using the optimized crowd counting model to obtain the real-time crowd counting information; The mobile client module is used to provide users with a mobile APP client. The mobile APP client includes a user interaction interface. The user interaction interface is used to display the crowd quantity information, queue management information and heat map of the crowd in the scene in real time in combination with the scene map.

2. The data collection device for crowd statistics based on AI technology according to claim 1 is characterized in that: The cameras in the acquisition module collect data from a variety of scenes including parks, playgrounds, shopping malls, school cafeterias and stations.

3. The data collection device for crowd statistics based on AI technology according to claim 2 is characterized in that: The interactive information in the auxiliary acquisition module includes one or more of video interactive information, image interactive information, advertisement interactive information and QR code interactive information.

4. The data collection device for crowd statistics based on AI technology according to claim 3 is characterized in that: The preprocessing operation in the data preprocessing module includes the following steps: Step 1: Data cleaning: removing redundant, repeated and invalid parts of the crowd density image data and interactive crowd density image data to obtain cleaned crowd density image data and interactive crowd density image data; Step 2, data denoising: using filtering technology to denoise the cleaned crowd density image data and interactive crowd density image data to obtain denoised crowd density image data and interactive crowd density image data; Step three, data enhancement: rotate, scale and flip the denoised crowd density image data and interactive crowd density image data to obtain pre-processed crowd density image data and interactive crowd density image data.

5. The data collection device for crowd statistics based on AI technology according to claim 4 is characterized in that: The model training module includes an accelerated training unit, which is used to use GPU and Ascend AI processor to accelerate the training of crowd statistics models.

6. The AI-based crowd statistics data collection device according to claim 5, characterized in that: The semantic segmentation algorithms in the model training module include one or more of the DeepLab algorithm and the Mask R-CNN algorithm.

7. The data collection device for crowd statistics based on AI technology according to claim 6 is characterized in that: The model optimization module includes a model evaluation unit, which is used to evaluate the recognition accuracy and counting error rate of the crowd statistics model on the validation set and generate evaluation results.

8. The data collection device for crowd statistics based on AI technology according to claim 7 is characterized in that: The model optimization module also includes a model optimization unit, which is used to adjust the network structure and learning rate of the convolutional neural network according to the evaluation results, and use the SGD algorithm to adjust the operating parameters and structure of the optimized crowd statistics model in real time.

9. The data collection device for crowd statistics based on AI technology according to claim 8, characterized in that: The crowd counting module includes an edge computing unit, which is used to analyze real-time video image data using an optimized crowd counting model deployed on a computing device near the large screen to obtain preliminary crowd count information.

10. The data collection device for crowd statistics based on AI technology according to claim 9, characterized in that: The crowd counting module also includes an end-cloud collaboration unit, which uses the data synchronization and transmission technology in the Ascend AI platform to upload preliminary crowd counting information to the cloud. The cloud then sends the preliminary crowd counting information to the model training module to assist in the training of the crowd statistics model.

Images