A dense crowd monitoring system based on a multi-level attention algorithm

Abstract

The application discloses a dense crowd monitoring system based on a multi-level attention algorithm. The crowd intelligent monitoring system is composed of a camera module, an embedded computing platform and a Web server. The camera module collects the crowd image information of a monitoring scene in real time and transmits the information to the embedded computing platform. The embedded computing platform carries a crowd intelligent recognition algorithm to analyze the crowd image information, transmits the obtained analysis graph to the Web server through a WIFI network for analysis, and obtains information such as quantity, positioning and density, which can be viewed by an administrator in the website. The website has a warning function, and when the analysis quantity exceeds a threshold value, the website will give a warning and the embedded device will play an aggregation alarm. The crowd intelligent recognition algorithm is a multi-level attention algorithm combined with channel attention, receptive field attention and spatial attention.

Description

Technical Field

[0001] This invention relates to a novel dense swarm monitoring technology, and more particularly to a dense swarm monitoring system based on a multi-level attention algorithm. Background Technology

[0002] Large gatherings of animals and objects can pose safety risks anytime, anywhere. Animal populations include not only groups of animals and objects, but also groups of people. Monitoring animal populations can provide timely information on the quantity and distribution of plants and animals in a scene, and can also monitor crowded areas to prevent safety hazards. Currently, society urgently needs a system capable of timely monitoring animal populations and providing prompt and effective early warnings.

[0003] Current cluster monitoring systems still rely on manual monitoring, which often involves subjective judgment and lacks objectivity. Furthermore, as the number of locations requiring monitoring increases, significant human resources are needed for monitoring and maintenance. In some scenarios, one person may need to monitor multiple locations simultaneously, which is a strain on their energy levels. Therefore, a dense cluster monitoring system has enormous potential for application.

[0004] With the breakthroughs and rapid development of artificial intelligence technologies centered on deep learning in recent years, the use of deep neural networks for image and even video processing has gained widespread acceptance in the industry. Introducing appropriate attention mechanisms into the computation of deep neural networks can make the neural network model more flexible and adaptable to various detection scenarios at the computational level. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies and improve the performance of existing methods, this invention proposes a dense swarm monitoring system based on a multi-level attention algorithm. It uses an edge-embedded device combined with a web cloud server to perform real-time swarm monitoring and can provide real-time early warning of swarm quantity by setting a threshold.

[0006] In the swarm monitoring algorithm, this invention proposes a neural network based on a multi-level attention method for embedded devices. It can intelligently analyze the input scene image and transmit the results to a web server via a WIFI network for parsing to obtain information such as quantity, location, and density, which can be viewed by the administrator on the website.

[0007] To achieve the above objectives, this invention proposes a dense cluster monitoring system based on a multi-level attention algorithm. The invention includes: a camera module, a speaker module, an embedded computing platform, a multi-level attention algorithm structure, and a web server. The camera module is used to acquire real-time images of the cluster and transmit them to the embedded computing platform. The speaker module is used to play cluster alarms to the dense cluster. The embedded computing platform is an ARM-based microcomputer motherboard running Linux, including but not limited to Raspberry Pi and Jetson Nano, and its main function is to house a neural network model and transmit the output results to the web server via Wi-Fi. The multi-level attention algorithm structure is a convolutional neural network model after deep learning, and its main function is to process the cluster images into analysis graphs. The web server is a remote computer running a website and processing analysis images, and its main function is to receive analysis graphs sent by the embedded computing platform, display the final processing results, and support the website's operation.

[0008] The embedded computing platform is an ARM-based microcomputer motherboard running Linux. After connecting to the camera module, the embedded computing platform reads the video captured by the camera module in real time. The platform extracts the current moment's image from the video by taking keyframes, normalizes the image, and then feeds it into a multi-level attention algorithm structure for image processing to obtain an analysis map. Finally, the analysis map is sent to a web server via Wi-Fi for further processing. If the web server returns an alert command, a gathering alarm is played to the dense group of objects via a speaker module.

[0009] The multi-level attention algorithm structure is a convolutional neural network architecture after deep learning, and the steps of this architecture are as follows:

[0010] First, the high-dimensional feature map is divided into different convolutional computation branches on an average basis according to each channel, and then the results from different branches are stacked. Next, the stacked feature map is shuffled using a feature shuffling method, and then one-dimensional convolution is used to extract receptive field attention and spatial attention from the shuffled feature map. Then, a new branch is started to extract spatial attention using two-dimensional convolution, and this is combined with the feature map from the previous step to obtain a feature map containing receptive field, channels, and spatial attention. Finally, this feature map is mapped to the final output analysis map using the sigmoid function.

[0011] The web server is a remote computer that runs the website and processes and analyzes images. Its main functions are to receive analysis graphs sent by the embedded computing platform, display the final processing results, and support the website's operation. The web server receives analysis graphs sent by the embedded computing platform through a network interface and parses the analysis graphs to obtain information on the quantity, location, and density of objects in the scene image.

[0012] Furthermore, by classifying each pixel in the analysis image, the quantity information of the population and its corresponding positional distribution information in the analysis image can be obtained; and after processing the analysis image with Gaussian blur with a Gaussian kernel of 15, a heat map of population density distribution can be generated using image heat map display.

[0013] The web server then displays the quantity, location, and density information of the scene image swarm obtained from the parsed analysis graph on a webpage, which can be viewed by the administrator at any time. If the quantity exceeds the threshold set by the administrator, an early warning will be issued and an early warning command will be sent to the embedded platform. The embedded platform will then play a gathering alarm to the swarm through the speaker module.

[0014] Compared with the prior art, the beneficial effects of the present invention are:

[0015] (1) The dense swarm monitoring system based on multi-level attention algorithm of the present invention has the advantages of low cost, saving manpower, high accuracy, high intelligence, beautiful effect and strong timeliness compared with the existing swarm monitoring system.

[0016] (2) This invention designs a multi-level attention algorithm by combining deep learning methods in the field of artificial intelligence to perform swarm analysis. It can automatically detect the number, location, density distribution and other comprehensive swarm reference information without manual counting, which greatly saves human resources and improves the efficiency of swarm supervision.

[0017] (3) The present invention designs a group monitoring system, which rationally allocates computing resources through edge and cloud collaborative processing, reducing resource waste; it can automatically detect images and display the results on the website, lowering the threshold for use and improving the detection quality; the reference quantity threshold can provide timely early warning and broadcast aggregation alarms to the group, ensuring the timeliness of management. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the architecture of a dense swarm monitoring system based on a multi-level attention algorithm according to an embodiment of the present invention;

[0020] Figure 2This is a flowchart illustrating a dense swarm monitoring system based on a multi-level attention algorithm according to an embodiment of the present invention.

[0021] Figure 3 This is a schematic diagram of the architecture of a multi-level attention algorithm for a dense swarm monitoring system based on a multi-level attention algorithm according to an embodiment of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] like Figure 1 The above is a schematic diagram of the architecture of a dense swarm monitoring system based on a multi-level attention algorithm, including: a camera module, a speaker module, an embedded computing platform, a multi-level attention algorithm structure, and a web server.

[0024] like Figure 2 The diagram illustrates a flowchart of a dense swarm monitoring system based on a multi-level attention algorithm. After connecting to a camera module, the embedded computing platform reads the video captured by the camera module in real time. The embedded computing platform extracts the current moment's image from the video by extracting keyframes, normalizes the image, and then feeds it into the multi-level attention algorithm structure for image computation to obtain an analysis graph.

[0025] like Figure 3 As shown, the multi-level attention algorithm structure is a convolutional neural network model after deep learning, and its main function is to process the object group image into an analysis map. Furthermore, the multi-level attention algorithm is a fully convolutional architecture, and the steps of this architecture are as follows:

[0026] S1, the high-dimensional feature map A extracted from the image is divided into four parts according to the number of channels. Each feature map corresponds to a dilated separable convolution with a dilation rate of 1, 2, 4, and 6, respectively. Furthermore, dilated separable convolutions with different dilation rates allow the convolutional feature extraction operation to obtain different receptive fields, and separable convolutions can reduce the number of computational parameters. Using these dilated separable convolutions with different dilation rates, features are further extracted from each of the segmented feature maps, resulting in four new feature maps convolved with different receptive fields. Finally, the four new feature maps are stacked, and their channels are uniformly shuffled to ensure uniform fusion, resulting in the high-dimensional feature map B. The dilated convolution operation process is as follows:

[0027]

[0028] Where y is the output of the dilated convolution, H and W are the length and width of the feature map, respectively, r is the dilation rate of the dilated convolution, and k is the convolution filter.

[0029] S2, global average pooling is used on the high-dimensional feature map B to map the feature information from three-dimensional space to one-dimensional space. Then, a one-dimensional convolution with a kernel size of 5 is used to extract attention from this one-dimensional vector. This one-dimensional convolution can consider both channel correlation and the importance of different receptive field features. Finally, a set of one-dimensional feature weights is obtained, and these one-dimensional feature weights are restored to three-dimensional space. The obtained three-dimensional feature weights are then fused with the high-dimensional feature map B to obtain the high-dimensional feature map C. This high-dimensional feature map C incorporates receptive field attention and channel attention. The process of extracting the one-dimensional feature weights is as follows:

[0030] K(B)=c 5 {avgpool[shuffle(B)]}

[0031] Where K is the one-dimensional feature weight (i.e., the receptive field attention and channel attention weights), B is the high-dimensional feature map input, avgpool is global average pooling, shuffle is uniform channel shuffling, and c 5 This is a one-dimensional convolution with a kernel size of 5.

[0032] S3, a new branch is opened on the high-dimensional feature map A to perform global max pooling and global average pooling, resulting in two two-dimensional feature maps containing different spatial information. These two two-dimensional feature maps are stacked, and spatial attention is extracted from the stacked two-dimensional feature map through a two-dimensional convolution with a kernel size of 7, resulting in a spatial feature weight. Finally, this spatial feature weight is fused with the high-dimensional feature map C to obtain the high-dimensional feature map D. This high-dimensional feature map D incorporates multi-level attention, including receptive field attention, channel attention, and spatial attention. The process of extracting the spatial attention weight is as follows:

[0033] M(A) = c 7x7 ([avgpool(A); maxpool(A)])

[0034] Where M is the spatial attention weight, A is the high-dimensional feature map input, avgpool is the global average pooling, maxpool is the global max pooling, and c 7×7 This is a two-dimensional convolution with a kernel size of 7.

[0035] S4, the high-dimensional feature map D is transformed into a two-dimensional space through a linear transformation, and then passed through a Sigmoid function to obtain an analysis map. This analysis map contains the probability value of each pixel, where the Sigmoid function is the final classifier. This detection information map is the analysis map output by this multi-level attention algorithm structure. The Sigmoid function is:

[0036]

[0037] like Figure 2 As shown, after obtaining the analysis graph, the embedded computing platform sends the analysis graph to the web server for further processing via WIFI data transmission.

[0038] like Figure 2 As shown, the web server receives the analysis graph sent by the embedded computing platform through the network interface, and parses the analysis graph to obtain the quantity, location, and density information of the objects in the scene image.

[0039] Furthermore, by classifying the probability of each pixel in the analysis image and calculating the number of pixels of the same category in the analysis image, the quantity information of the group can be obtained; by observing the distribution of pixels of the same category in the analysis image, the distribution information of the group can be obtained; and after processing the analysis image with Gaussian blur with a Gaussian kernel of 15, a heat map of the group density distribution can be generated using the image heat map display method.

[0040] like Figure 2 As shown, the web server then displays the quantity, location, and density information of the scene image swarm obtained from the parsing and analysis graph on a webpage, which can be viewed by the administrator at any time. If the quantity exceeds the threshold set by the administrator, an early warning will be issued and an early warning command will be sent to the embedded platform. The embedded platform will then play a gathering alarm to the swarm through the speaker module.

[0041] The above provides a detailed description of a dense swarm monitoring system based on a multi-level attention algorithm provided by the embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A dense swarm monitoring system based on a multi-level attention algorithm, characterized in that, The dense swarm monitoring system includes: The camera module captures real-time images of objects in the monitored scene and transmits them to the embedded computing platform. The embedded computing platform analyzes the object image information using a multi-level attention algorithm to obtain an analysis map containing object information, and then sends the obtained analysis map to the web server via a WIFI network. The multi-level attention algorithm is a fully convolutional architecture, which processes the group image into an analysis map; The web server parses the data in the analysis graph to obtain quantity, location, and density information, and displays it on the webpage. If the quantity exceeds the threshold set by the website, the alert function is activated, the website issues an alert to notify the administrator, and sends an alert command to the embedded computing platform. The embedded computing platform receives warning commands from the web server and broadcasts aggregation alarms to the swarm via a speaker module. The multi-level attention algorithm divides the feature map into multi-branch convolutions by channel. The results from different branches are stacked and the features are shuffled. One-dimensional convolution is used to extract the receptive field and channel-based hybrid attention, and two-dimensional convolution is used to fuse spatial attention, including: The high-dimensional feature map A extracted from the image is divided into four parts according to the number of channels. Each feature map corresponds to a dilated separable convolution with a dilation rate of 1, 2, 4, and 6, respectively. Different dilated separable convolutions with different dilation rates allow the convolutional feature extraction operation to obtain different receptive fields. These dilated separable convolutions with different dilation rates are used to re-extract features from each segmented feature map, resulting in four new feature maps with different receptive fields. Finally, the four new feature maps are stacked, and their channels are uniformly shuffled to ensure uniform fusion, resulting in the high-dimensional feature map B. The dilated convolution operation process is as follows: Where y is the output of the dilated convolution, H and W are the length and width of the feature map, respectively, r is the dilation rate of the dilated convolution, and k is the convolution filter; The web server is a remote computer running the website and processing and analyzing images. The analysis image is a two-dimensional data image containing probability information for each pixel after being processed by the Sigmoid function. The data parsing process includes: The probability of each pixel in the analysis image is used for classification, the number of pixels of the same category in the analysis image is calculated, and the quantity information of the group is obtained; the distribution of pixels of the same category in the analysis image is observed to obtain the distribution information of the group; and after the analysis image is processed by Gaussian blur with a Gaussian kernel of 15, a heat map of the density distribution of the group is generated using image heat map display method. The aforementioned early warning functions include website early warning reports, sending early warning commands from the web server, and playing clustered alarms on the embedded computing platform.

2. The dense swarm monitoring system based on a multi-level attention algorithm as described in claim 1, characterized in that, The embedded computing platform is an ARM-based microcomputer motherboard running a Linux system.

Images