Real-time crowd flow monitoring and early warning method and device for scenic spots

Abstract

The present invention discloses a real-time crowd flow monitoring and early warning method and device for scenic spots, which relates to the field of data analysis technology. The method comprises: monitoring the video stream of the target area through an image sensor array, constructing an analysis model including personnel detection and target tracking, performing personnel detection and target tracking, and obtaining relevant results. Based on these results, the flow of people is counted, and the regional crowd flow density distribution map is displayed through a heat map, and high-density areas are identified and flow control is implemented. The technical problem that the existing crowd flow monitoring system in scenic spots cannot monitor in real time and accurately, and cannot control high-density crowd flow in time is solved, and the technical effect of real-time and accurate monitoring of the crowd flow in scenic spots and improving the flow management capability of scenic spots is achieved.

Description

Technical Field

[0001] The present application relates to the field of data analysis technology, and in particular to a real-time crowd flow monitoring and early warning method and device for scenic spots. Background Art

[0002] With the increasing number of tourists in scenic spots, especially during holidays and peak hours, dense tourists often bring safety hazards, traffic jams and reduced tourist experience. Traditional methods of monitoring the flow of people in scenic spots usually rely on manual patrols or basic sensor data, which have defects such as untimely information acquisition, poor accuracy and lack of dynamic adjustment capabilities, and cannot effectively respond to the ever-changing tourist flow. The existing system fails to achieve real-time monitoring and precise control of the flow of people in scenic spots, and lacks timely identification and emergency response to high-density areas. Therefore, there is an urgent need for a solution that can accurately monitor and dynamically respond to changes in the flow of people in scenic spots in real time to ensure that the scenic spot can efficiently manage the flow of tourists and take timely measures to prevent potential risks brought by high-density people.

[0003] Among the current relevant technologies, there are technical problems that the scenic area crowd flow monitoring system cannot monitor in real time and accurately, and cannot control high-density crowd flow in a timely manner. Summary of the invention

[0004] The present application solves the technical problem that the existing crowd monitoring system in scenic spots cannot conduct real-time and accurate monitoring and cannot timely control high-density crowd flow by providing a real-time crowd monitoring and early warning method and device for scenic spots.

[0005] This application provides a real-time crowd flow monitoring and early warning method for scenic spots, including:

[0006] The target area is monitored by video through an image sensor array to obtain video stream data within a predetermined time window; a crowd flow analysis model is constructed, wherein the crowd flow analysis model includes a personnel detection unit and a target tracking unit; the video stream data is input into the crowd flow analysis model, personnel detection and target tracking are performed through the personnel detection unit and the target tracking unit, and personnel detection results and target tracking results are obtained; crowd flow statistics are performed based on the personnel detection results and the target tracking results, and the crowd flow statistics results are visualized through a heat map to generate a regional crowd flow density distribution map; the regional crowd flow density distribution map is identified, and a high-density area is determined for flow limiting control.

[0007] This application also provides a real-time crowd flow monitoring and early warning device for scenic spots, including:

[0008] A video monitoring module, the video monitoring module is used to perform video monitoring of the target area through an image sensor array to obtain video stream data within a predetermined time window; a crowd analysis model construction module, the crowd analysis model construction module is used to construct a crowd analysis model, wherein the crowd analysis model includes a personnel detection unit and a target tracking unit; a result acquisition module, the result acquisition module is used to input the video stream data into the crowd analysis model, perform personnel detection and target tracking through the personnel detection unit and the target tracking unit, and obtain personnel detection results and target tracking results; a crowd flow statistics module, the crowd flow statistics module is used to perform crowd flow statistics based on the personnel detection results and the target tracking results, visualize the crowd flow statistics results through a heat map, and generate a regional crowd flow density distribution map; a flow limiting control module, the flow limiting control module is used to identify the regional crowd flow density distribution map, and determine the high-density area for flow limiting control.

[0009] The real-time crowd flow monitoring and early warning method and device for scenic spots proposed in this application first monitors the video stream of the target area through an image sensor array, builds an analysis model including personnel detection and target tracking, performs personnel detection and target tracking, and obtains relevant results. Based on these results, the flow of people is counted, and the regional crowd density distribution map is displayed through a heat map, high-density areas are identified and flow control is implemented, achieving the technical effect of real-time and accurate monitoring of the flow of people in the scenic area and improving the flow management capability of the scenic area. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the accompanying drawings of the embodiment of the present invention will be briefly introduced below. A flow chart is used in the present application to illustrate the operations performed by the device according to the embodiment of the present application. It should be understood that the previous or following operations are not necessarily performed precisely in order. On the contrary, various steps can be processed in reverse order or simultaneously as needed. At the same time, other operations can also be added to these processes, or one or more operations can be removed from these processes.

[0011] Figure 1 A flow chart of a real-time crowd flow monitoring and early warning method for a scenic spot provided in an embodiment of the present application;

[0012] Figure 2 A schematic diagram of the structure of a real-time crowd flow monitoring and early warning device for a scenic spot provided in an embodiment of the present application.

[0013] Explanation of the reference numerals: video monitoring module 10 , crowd flow analysis model building module 20 , result acquisition module 30 , crowd flow statistics module 40 , flow limiting control module 50 . DETAILED DESCRIPTION

[0014] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below.

[0015] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0016] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict, and the terms "first\second" involved are merely to distinguish similar objects and do not represent a specific ordering of objects. The terms "including" and "having" and any variations are intended to cover non-exclusive inclusions, for example, a process, method, device, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products, or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by technicians in the technical field of this application. The terms used herein are for the purpose of describing the embodiments of the present application only.

[0017] The present application embodiment provides a real-time crowd flow monitoring and early warning method for a scenic spot, such as Figure 1 As shown, the method includes:

[0018] Step S100, video monitoring of the target area is performed through the image sensor array to obtain video stream data within a predetermined time window. Specifically, after a comprehensive survey of the target area, the camera installation position is determined, and the camera is installed at key locations such as entrances, scenic spots, and passages to ensure stability and appropriate angles. Then, the resolution, frame rate, sensitivity and other parameters are set for each camera, and the sensitivity is reduced in areas with sufficient light, and the light is adjusted appropriately in dark areas. Then, the camera is started to collect the video stream data after the optical signal conversion process, and the data is continuously acquired according to the predetermined time window, and transmitted to the data processing center via wired or wireless means. During the transmission, data verification and error correction technology is used to ensure completeness and accuracy. Pre-processing such as data format conversion, image enhancement processing and cache management are performed in the processing center to lay the foundation for subsequent processing.

[0019] In a possible implementation, the target area is monitored by video through an image sensor array to obtain video stream data within a predetermined time window, and step S100 further includes step S110, wherein the image sensor array is used to cover the target area in full view, including N cameras, wherein each camera corresponds to a predetermined installation position, and N is an integer greater than 1. Specifically, the target area is surveyed and analyzed on the spot, considering its geographical form, personnel flow pattern, and visual blind spots, and then the camera layout is carefully designed based on these results, and the cameras are reasonably arranged at the entrance, square, passage, personnel gathering hotspots, and blind spots. Then, installation preparations are made for each predetermined installation position, the structural stability is checked, environmental factors are considered, and the camera is firmly and accurately installed and debugged with professional tools to ensure its normal operation. Finally, the installation and debugging steps are repeated as the number of cameras N increases to ensure that the image sensor array is effective and reliable.

[0020] Step S200, construct a crowd flow analysis model, wherein the crowd flow analysis model includes a personnel detection unit and a target tracking unit. Specifically, it is clarified that the crowd flow analysis model is used for personnel detection and tracking, and an architecture including a personnel detection unit and a target tracking unit and determining their relationship is designed. The personnel detection unit selects the YOLOX algorithm, and the network structure is designed according to the characteristics of the target area. Multiple sample images are collected and features are annotated before training, and evaluation and optimization are performed during training. The target tracking unit conducts in-depth research on the principles of the selected algorithm, and an interface with the personnel detection unit is designed to ensure collaboration. The tracking performance and stability are optimized according to the scenario, and the overall performance of the model is improved to meet actual needs.

[0021] In a possible implementation, a crowd flow analysis model is constructed, wherein the crowd flow analysis model includes a personnel detection unit and a target tracking unit, and step S200 further includes step S210, collecting a sample monitoring image set, and marking the personnel features in different sample monitoring images to obtain a sample personnel feature set. Specifically, according to the actual situation of the target area, the scene range of collecting sample monitoring images is determined. If the target area is a scenic area, the collection scene will cover various areas of the scenic area, including entrances, exits, various scenic spots, walkways, rest areas, dining areas, etc. Collection is performed in different time periods, such as early morning, noon, evening, night, etc., to obtain images under different lighting conditions, and different weather conditions, such as sunny days, cloudy days, rainy days, foggy days, etc., are considered to ensure that the sample can reflect the impact of various environmental factors on personnel detection. Use image acquisition equipment, such as high-resolution digital cameras, high-definition cameras, etc., to collect images in the selected scene according to a predetermined acquisition strategy. The acquisition strategy includes the diversification of shooting angles, shooting images of people's activities from different angles such as looking straight, looking down, and looking up; the change of shooting distance, including both close-up images of people's detailed features and long-distance images of people's group distribution. In addition, we also focus on collecting images with different density of people, from scenes with sparse personnel to scenes with dense and crowded personnel, to ensure that the sample monitoring image set has sufficient diversity and representativeness. After collecting the sample monitoring image set, detailed rules for marking personnel characteristics are formulated. The marked personnel characteristics mainly include the position, outline, posture, clothing, facial features (if applicable), etc. of the personnel. For the location of people, the center coordinates of the people in the image and the approximate boundaries of their range of activities are marked; contour marking determines the shape of people by outlining the edges of their bodies; posture marking is used to describe different states of people, such as standing, walking, sitting, and running; clothing marking records the color, style, and other information of people's clothing, so that they can be distinguished by clothing features in subsequent detection; if the facial features are clearly visible and helpful for detection, such as in the scene of face recognition at the entrance of a scenic spot, the key facial organs such as eyes, nose, and mouth of people are marked, and the marking software tools are used to carefully mark the features of people in different sample monitoring images according to the established marking rules to ensure the accuracy and consistency of the marking. In the marking process, for difficult-to-determine features or ambiguous situations, they are resolved through group discussions or reference to expert opinions. Each sample monitoring image must be checked by multiple people to ensure that the marking is correct, and the marked personnel feature information is organized into a sample personnel feature set to provide an accurate data basis for subsequent model training.

[0022] Step S220, construct the network structure of the personnel detection unit based on the YOLOX target detection algorithm, use the sample monitoring image set and the sample personnel feature set as training data, train the personnel detection unit until convergence, and obtain a personnel detection unit that meets the expected conditions. Specifically, analyze the principles and architectural features of the YOLOX target detection algorithm, and construct the network structure of the personnel detection unit according to the needs of personnel detection in the target area. YOLOX uses technologies such as decoupling head design, data enhancement strategy, and optimized loss function. When constructing the network, it determines the parameters and connection methods of each layer of the network, such as the input layer, convolution layer, pooling layer, and fully connected layer. For example, according to the size of the input image, set appropriate input layer parameters, select the appropriate number and size of convolution kernels for feature extraction, determine the step size and window size of the pooling layer to reduce the data dimension, design the number of neurons in the fully connected layer to adapt to the output format of the personnel detection results, including the category of personnel (if there are multiple types of personnel to be distinguished), location coordinates, confidence and other information, use the collected sample monitoring image set as input data, and the sample personnel feature set as the target output data to train the constructed personnel detection unit. During the training process, the sample monitoring image set is first preprocessed, including image scaling, cropping, normalization and other operations to make it meet the requirements of network input. Then, the preprocessed image is input into the network, and the output prediction value of the network is calculated by forward propagation. Next, the loss function is calculated based on the difference between the predicted value and the true labeled value in the sample personnel feature set. Using the back propagation algorithm, the weights and parameters of the network are adjusted according to the gradient information of the loss function, so that the prediction results of the network gradually approach the true value. Repeat this process and continuously iterate the training until the network converges, that is, the value of the loss function no longer decreases significantly. At this time, the obtained personnel detection unit can accurately detect people in the target area to a certain extent, meet the expected detection performance requirements, and provide reliable personnel detection capabilities for subsequent practical applications.

[0023] In a possible implementation, a crowd flow analysis model is constructed, wherein the crowd flow analysis model includes a personnel detection unit and a target tracking unit, and step S200 further includes step S230, wherein the target tracking unit is constructed based on the ByteTrack tracking algorithm, wherein the target tracking unit includes an input layer and an output layer, wherein the input data of the input layer is personnel features, and the output data of the output layer is a target ID and a target motion trajectory, wherein each personnel feature has a uniquely identified target ID. Specifically, the association mechanism of the ByteTrack tracking algorithm is analyzed, and its principle of matching personnel features between different frames and handling complex situations such as occlusion is understood, then the input layer of the target tracking unit receives and pre-processes the personnel feature data transmitted by the personnel detection unit to ensure that the format quality meets the requirements, the output layer assigns a unique ID to the new target and records and updates the motion trajectory, optimizes the trajectory and calculates related parameters, and then integrates the target tracking unit with the personnel detection unit to achieve collaborative work, and finally adjusts the association threshold, the tracker life cycle and other parameters according to the scene requirements, and uses auxiliary technology to optimize the performance, so as to provide a reliable tracking service for the system.

[0024] Step S300, input the video stream data into the crowd flow analysis model, perform personnel detection and target tracking through the personnel detection unit and the target tracking unit, and obtain the personnel detection results and target tracking results. Specifically, after obtaining the video stream data from the image sensor array, it is necessary to pre-process it first, including converting it into a model-suitable format and extracting frame images, normalizing the images, and adjusting the resolution and size according to the model requirements. Then, the personnel detection unit of the crowd flow analysis model is input frame by frame. The unit extracts personnel features based on the convolution layer, and then the classifier and regressor determine the personnel category, position and confidence. If a person is detected, the current frame image and the personnel detection result are transmitted to the target tracking unit. This unit calculates the feature similarity based on the ByteTrack algorithm and compares it with the threshold to determine the association relationship, assigns an ID to the new target and records the motion trajectory. Finally, the personnel detection and target tracking results are integrated, stored in a structured format or transmitted to other modules for scenic area personnel flow statistics and personnel dynamic display, etc., to help managers make decisions.

[0025] In a possible implementation, the video stream data is input into the crowd analysis model, and the personnel detection and target tracking are performed by the personnel detection unit and the target tracking unit to obtain the personnel detection results and the target tracking results. Step S300 further includes step S310, in which the personnel detection unit performs personnel detection on the video stream data and outputs the personnel detection results. Specifically, the personnel detection unit receives the video stream data obtained from the image sensor array and preprocessed (such as format conversion, normalization, resolution adjustment, etc.), and the video stream data enters the personnel detection unit in the form of frame-by-frame images. While receiving the data, the personnel detection unit initializes its internal detection-related parameters, such as network weights, threshold settings, etc., to ensure the accuracy and stability of the detection process. For example, according to the optimal parameter combination determined when the model was trained before, the corresponding weight file is loaded to put the network in the best detection state. The personnel detection unit uses its pre-built network structure based on the YOLOX target detection algorithm to process the input video stream frame image. The convolution layer of the network begins to perform convolution operations on the image, extracting various features in the image, such as low-level features such as edges, textures, and colors, and performs dimensionality reduction processing through the pooling layer, gradually forming high-level features that are more representative of people. Feature information is transmitted layer by layer in the network and finally reaches the fully connected layer. The fully connected layer conducts a comprehensive analysis of the extracted features based on the patterns learned in the previous training to determine whether there are people in the image and the approximate location of the people. For example, by comparing the features with the human feature patterns in the training data, it is determined whether certain areas in the image have characteristic expressions of people, such as human body contours, clothing features, etc. After the network's analysis and judgment, the person detection unit outputs the person detection results. The detection results include multiple aspects of information. The first is the number of people detected, which clearly identifies how many people are identified in the current frame image. The second is the location information of each person, which is given in the form of a bounding box. The coordinates of the bounding box accurately define the position range of the person in the image, such as the upper left corner coordinates and the lower right corner coordinates. In addition, the confidence of each detection result is given. The confidence indicates the reliability of the detection result. The value range is usually between 0 and 1. The closer to 1, the more reliable the detection result is, and the closer to 0, the greater the possibility of error or misjudgment. For example, if three people are detected in a frame, the bounding box coordinates and corresponding confidence values ​​of each person are given. For example, the bounding box coordinates of person 1 are (x1, y1, x2, y2), and the confidence is 0.85; the bounding box coordinates of person 2 are (x3, y3, x4, y4), and the confidence is 0.92; the bounding box coordinates of person 3 are (x5, y5, x6, y6), and the confidence is 0.78.

[0026] Step S320, if the personnel detection result is not 0, the video stream data and the personnel detection result are transferred to the target tracking unit, the target is continuously tracked, and the target tracking result is output. Specifically, when the personnel detection result output by the personnel detection unit is not 0, that is, when at least one person is detected, the current video stream data (current frame image) and the personnel detection result (including the number of personnel, position, confidence, etc.) are transferred to the target tracking unit. After receiving these data, the target tracking unit first initializes the tracking-related parameters, such as the target ID counter (used to assign a unique ID to the newly detected target), the tracking trajectory data structure (used to record the motion trajectory of the target), the internal state of the association algorithm, etc. For example, the target ID counter is initialized to 1, and the ID of 1 is assigned to the first newly detected target. Based on the ByteTrack tracking algorithm, the target tracking unit uses the personnel feature information (such as position, appearance features, etc.) in the input personnel detection result to find the association with the tracked target in the previous frame in the current frame. By calculating the similarity between features, a suitable similarity measurement method is used, such as a distance measurement based on position, a cosine similarity based on appearance features, etc., and compared with a preset association threshold. If the calculated similarity is higher than the association threshold, it is considered that the person in the current frame is the same target as a target in the previous frame, thereby achieving continuous tracking of the target; if the similarity is lower than the threshold, it may indicate that a new target has appeared or the previous target has been lost (such as due to occlusion, etc.). For the case where it is determined to be the same target, the target tracking unit updates the motion trajectory information of the target and adds the position information of the target in the current frame to its motion trajectory data structure to record the continuous motion path of the target. For example, if the position of target 1 in the previous frame is (x7, y7) and the position in the current frame is (x8, y8), then (x8, y8) is added to the motion trajectory of target 1 to form a trajectory sequence such as [(x7, y7), (x8, y8)]. For the case where it is determined to be a new target, the target tracking unit assigns it a new, unique target ID, and then starts recording its motion trajectory, starting from the position where it was first detected. For example, a new person is detected and assigned a target ID of 4, and its initial position (x9, y9) is recorded as the starting point of its motion trajectory. The target tracking unit outputs the target tracking result, which mainly includes the unique identifier of each target (target ID) and its corresponding motion trajectory information. The target ID is used to distinguish different target individuals during the entire tracking process. No matter how the target moves, whether it is blocked or temporarily disappears, its ID remains unchanged, ensuring the continuity and identifiability of the tracking. The motion trajectory information records the motion path of the target in the video stream in detail, and displays the target's movement process through a series of position points (such as coordinates at different time points).For example, the ID of output target 1 is 1, and the motion trajectory is [(x10, y10), (x11, y11), (x12, y12)] (indicating the position change of target 1 in different frames); the ID of target 2 is 2, and the motion trajectory is [(x13, y13), (x14, y14)], etc. These results can be further used for subsequent processing such as crowd flow statistics, personnel behavior analysis, and data visualization, providing key target tracking data support for the entire system.

[0027] Step S400, perform crowd statistics based on the personnel detection results and target tracking results, visualize the crowd statistics results through heat maps, and generate regional crowd density distribution maps. Specifically, determine the spatiotemporal scope of crowd statistics based on the actual layout of the target area and monitoring needs, such as dividing scenic spots by opening hours and geographical areas, and dividing them into multiple sub-areas at the same time. Then, by relying on the number of personnel and location information of the personnel detection unit and the target ID and motion trajectory information of the target tracking unit, associate the personnel position with the sub-area, and build a model to count the number of people entering and leaving the sub-area at time intervals. Then, deeply understand the principle of heat maps, set parameters such as color mapping, normalization method and resolution according to the target area and visualization requirements, and map the crowd flow data to color values ​​to generate heat maps. After that, obtain the map information of the target area, merge the heat map with the map, and adjust the transparency and position to make the two correspond. Finally, the regional crowd density distribution map is displayed through a visualization platform, which can be viewed in real time or historically. Scenic area managers can use this to understand the distribution and flow trends of people, regulate crowded areas during peak hours, and optimize facility layouts. The system also ensures stable and accurate detection and tracking due to its relevant characteristics and mechanisms, and adapts to the scenic area environment, providing strong support for scenic area management and operations.

[0028] In a possible implementation, the flow of people is counted according to the personnel detection results and the target tracking results, and the flow of people statistics is visualized through a heat map to generate a regional flow density distribution map; step S400 further includes step S410, according to a predetermined division rule, the target area is divided to obtain a plurality of target sub-areas, wherein each target sub-area is marked with a regional area. Specifically, a predetermined division rule is formulated according to the characteristics of the target area and the monitoring requirements. If the target area is a scenic area, the division rule may be based on the functional areas of the scenic area (such as scenic spots, rest areas, dining areas, shopping areas, etc.), geographical features (such as hillsides, lakes, wooded areas, etc.) and personnel flow patterns (such as main channels, popular tourist routes, etc.). For example, each independent scenic spot in the scenic area is divided into a separate sub-area, and the main pedestrian path connecting the scenic spots is divided into different sub-areas at intervals of a certain length to ensure that each sub-area has relative independence and consistency in function and personnel flow. According to the formulated division rule, the target area is accurately divided to obtain a plurality of target sub-areas. During the segmentation process, use geographic information system (GIS) technology or professional image segmentation tools to ensure the accuracy and clarity of the sub-area boundaries. Each target sub-area is given a unique identifier and its area is recorded. For example, for a scenic spot sub-area with an area of ​​1,000 square meters, it is marked as "Scenic spot A area, area 1,000 square meters" so that the data of the sub-area can be accurately identified and processed in subsequent crowd flow statistics and analysis.

[0029] Step S420, according to the personnel detection result and the target tracking result, the number of people in the video stream data respectively in the multiple target sub-areas is counted to obtain the number of people in multiple areas. Specifically, based on the personnel detection result (including detailed information such as the number of people, position, etc.) output by the personnel detection unit and the target tracking result (covering the target ID and motion trajectory) output by the target tracking unit, the position information of the personnel in the video stream is associated with the divided target sub-areas. Through the motion trajectory in the target tracking result, the specific sub-area where each person is located at different time points is determined. For example, when a tourist's motion trajectory shows that it enters from one sub-area to another sub-area, the system can accurately record its transfer process, and according to the association between the personnel and the sub-area, the number of people in the video stream data respectively in multiple target sub-areas is counted. In each time interval (such as every minute or every 5 minutes), the number of people in each sub-area is counted. For example, at a certain moment, the personnel detection unit in the scenic spot area A detects that there are 50 people in the area. After further confirmation and association by the target tracking unit, it is finally determined that the number of people in the scenic spot area A at this moment is 50 people, and the number of people in the area corresponding to the sub-area is obtained.

[0030] Step S430, calculate the crowd density of multiple sub-regions according to the crowd numbers of the multiple regions and the areas of the multiple regions. Specifically, a sub-region crowd density calculation model is constructed based on the crowd numbers of the multiple regions and the areas of the multiple regions obtained above. The calculation formula of the model is: crowd density = number of regional crowds / regional area. For example, for the scenic spot area A mentioned above, if its area is 1,000 square meters and the number of people is 50, then the crowd density of scenic spot area A = 50 / 1000 = 0.05 people / square meter. Applying this calculation model, calculate the crowd density of each target sub-region in turn to obtain crowd density data of multiple sub-regions. These data reflect the density of people in different sub-regions and provide key data support for subsequent visualization and analysis.

[0031] Step S440, visualize the crowd density of the multiple sub-areas through a heat map to generate a regional crowd density distribution map. Specifically, the generation principle of the heat map is deeply understood, that is, the size of the data value is intuitively represented by the change of color depth. In this solution, the darker the color, the higher the crowd density of the sub-area, and the lighter the color, the lower the crowd density. According to the actual situation and visualization requirements of the target area, set appropriate heat map parameters, such as color mapping scheme (select a gradient color mapping from light blue to dark red, light blue indicates low crowd density, and dark red indicates high crowd density), data normalization method (ensure that the crowd density data of different sub-areas can be accurately displayed within a suitable color range) and the resolution of the heat map (according to the size and accuracy requirements of the target area, select a suitable resolution, such as for large scenic spots, a lower resolution can be used to improve calculation efficiency, and for key areas of concern, a high resolution can be used to display more details). The calculated crowd density data of multiple sub-areas are mapped to the heat map. According to the set color mapping scheme and data normalization method, the crowd density value of each sub-area is converted into a corresponding color value. For example, a sub-area with a crowd density of 0.05 people / square meter may be mapped as light blue, while a sub-area with a crowd density of 0.5 people / square meter may be mapped as dark blue. Through such a mapping process, the abstract crowd density data is converted into intuitive color information to generate a heat map. The generated heat map is fused with the map information of the target area to obtain a regional crowd density distribution map. During the fusion process, it is ensured that each sub-area in the heat map accurately corresponds to the actual area on the map of the target area, and the transparency and position of the heat map are adjusted so that it can clearly display the crowd density information without blocking important geographical signs and information on the map. For example, setting the transparency of the heat map to a moderate level allows managers to see the names of scenic spots, road directions and other information on the heat map and the map at the same time, so as to intuitively understand the distribution and flow trends of people in the scenic area, so as to make timely management decisions, such as taking flow control measures in crowded areas or guiding tourists to areas with less traffic, optimizing the allocation of scenic area resources, and improving tourists' tour experience.

[0032] Step S500, identifying the density distribution map of the regional flow of people, and determining the high-density area for flow control. Specifically, the image data of the regional flow density distribution map is obtained from the system, and after pre-processing such as reading and decoding and resolution adjustment, grayscale processing is performed to convert the color image into a grayscale image to highlight the brightness information. Then, the flow density threshold is set according to the carrying capacity of the scenic area, and the grayscale image is analyzed pixel by pixel, and the pixel grayscale value is compared with the threshold. The area where the pixel points above the threshold are located may be a high-density area, and a filtering algorithm is used to remove noise interference. After determining the high-density area, it is marked with a specific color or icon on the scenic area map or monitoring screen, and an electronic display screen or broadcasting system is set at the entrance and other locations to issue warning information to inform tourists. Subsequently, staff are arranged to manually limit the flow at the entrance of the flow-limited area, such as releasing in batches, and the release speed can be automatically adjusted according to the real-time flow density in combination with the intelligent gate. At the same time, the flow-limiting information is pushed to tourists through the official website of the scenic area and the mobile phone application to ensure the safe operation of the scenic area and the tourists' tour experience.

[0033] In a possible implementation, the regional crowd density distribution map is identified, and the high-density area is determined to perform flow control. Step S500 further includes step S510, and the regional crowd density distribution map is regionally screened according to a predetermined color index to determine multiple high-density areas. Specifically, the meaning and setting basis of the predetermined color index are clarified. In the regional crowd density distribution map, color is usually used to intuitively represent the size of the crowd density, and the predetermined color index is the screening standard determined according to the corresponding relationship between this color and density. For example, red and similar dark colors may be set to represent high-density areas, and blue and similar light colors may represent low-density areas. This setting is based on the intuitive perception of human vision on the depth of color and the amount of color, and dark colors often give people a denser feeling. Before performing regional screening, the regional crowd density distribution map needs to be loaded into the analysis system in a suitable image format (such as the common RGB format) to ensure that the image data is complete and the color information of each pixel can be accurately read. Perform pixel-by-pixel color analysis on the regional crowd density distribution map. The system reads the color value of each pixel and determines whether the area where the pixel is located belongs to a high-density area based on the predetermined color index. For example, for a heat map that uses a red-to-blue gradient color mapping to represent the density of human traffic, if the predetermined red (RGB value is within a certain range, such as R:200-255, G:0-50, B:0-50) indicates high density, then when the RGB value of a pixel is analyzed to be (230,30,20), it is preliminarily determined that the pixel may be in a high-density area. Considering that there may be color gradient transition areas or color deviations caused by drawing errors in the image, a reasonable color tolerance range is set. In the above example, if the tolerance is set to ±10, then pixels with RGB values ​​in the range of (220-240,20-40,10-30) may be considered to belong to the relevant range of the high-density area. In this way, the entire image is traversed to determine all possible high-density area pixels. Adjacent high-density area pixels are merged and divided into regions. Since the high-density area may be one or more continuous blocks in the image, these adjacent pixels need to be combined into a complete area. Image processing techniques such as region growing algorithms can be used to start from a determined high-density pixel and expand to the surrounding adjacent pixels. As long as the color of the adjacent pixels is within the tolerance range and is not marked as other regions, they will be included in the current high-density region. Repeat this process until all adjacent high-density pixels are merged into the corresponding region, and finally determine multiple high-density regions.

[0034] Step S520, according to the predetermined control scheme, the multiple high-density areas are subjected to flow control. Specifically, the specific content and execution steps of the predetermined control scheme are interpreted. The control scheme may include a variety of flow limiting measures and their implementation conditions, such as determining different flow limiting methods according to factors such as the size of the area of ​​the high-density area, the density of personnel, and the duration. At the same time, prepare the resources required for implementing flow limiting control, such as setting up sufficient signs (such as flow limiting signs, guide arrows, etc.) at the entrance of the scenic area and around the high-density area, arranging a sufficient number of staff, and ensuring that the intelligent flow limiting equipment (such as intelligent gates, flow counters, etc.) operates normally and is connected smoothly to the control system. For the multiple high-density areas determined, flow limiting control is performed according to the predetermined control scheme. If the control scheme stipulates that when the density of personnel in the high-density area exceeds a certain value (such as 1.5 people per square meter) and the area reaches a certain scale (such as more than 100 square meters), measures to restrict personnel entry are taken, then at the entrance of the corresponding area, the staff guides tourists to other areas based on the real-time flow density data counted by the intelligent device, and adjusts the release rules of the intelligent gate to reduce the flow of people entering the area. For example, the gate release interval is adjusted from 10 people per minute to 5 people per minute. During the flow control process, the changes in the density of people in high-density areas are continuously monitored. Through real-time video stream analysis and crowd statistics data updates, the effectiveness of flow control measures can be timely understood. If it is found that the density of people in a high-density area continues to rise or has no obvious downward trend after the flow is limited, the flow control measures will be further adjusted according to the control plan, such as increasing the guidance of staff, expanding the scope of flow control, or coordinating with other areas to guide people (such as guiding tourists from high-density areas to adjacent low-density areas), to ensure that the density of people in high-density areas can be gradually reduced to a safe range, to ensure the safety and sightseeing experience of people in the scenic area.

[0035] The embodiment of the present application uses a video stream of the target area monitored by an image sensor array to build an analysis model including personnel detection and target tracking, perform personnel detection and target tracking, and obtain relevant results. Based on these results, the flow of people is counted, and the regional flow density distribution map is displayed through a heat map, high-density areas are identified and flow control is implemented, achieving the technical effect of real-time and accurate monitoring of scenic area flow and improving the flow management capability of scenic areas.

[0036] In the above, refer to Figure 1 The real-time crowd flow monitoring and early warning method for a scenic spot according to an embodiment of the present invention is described in detail. Figure 2 A real-time crowd flow monitoring and early warning device for a scenic spot according to an embodiment of the present invention is described.

[0037] The real-time crowd flow monitoring and early warning device for scenic spots according to the embodiment of the present invention is used to solve the technical problem that the existing crowd flow monitoring system of scenic spots cannot monitor in real time and accurately, and cannot timely control the high-density crowd flow, and achieves the technical effect of real-time and accurate monitoring of the crowd flow in scenic spots and improving the flow management capability of scenic spots. The real-time crowd flow monitoring and early warning device for scenic spots includes: a video monitoring module 10, a crowd flow analysis model construction module 20, a result acquisition module 30, a crowd flow statistics module 40, and a flow limiting control module 50.

[0038] The specific configuration of the video monitoring module 10 will be described in detail below. As described above, the target area is monitored by the image sensor array to obtain video stream data within a predetermined time window. The video monitoring module 10 further includes: a full-view coverage unit, which is used for the image sensor array to cover the target area in full view, including N cameras, wherein each camera corresponds to a predetermined installation position, and N is an integer greater than 1.

[0039] The specific configuration of the crowd flow analysis model construction module 20 will be described in detail below. As described above, a crowd flow analysis model is constructed, wherein the crowd flow analysis model includes a personnel detection unit and a target tracking unit, and the crowd flow analysis model construction module 20 further includes: a feature annotation unit, the feature annotation unit is used to collect a sample monitoring image set, and annotate personnel features in different sample monitoring images to obtain a sample personnel feature set; a feature set training unit, the feature set training unit is used to construct a network structure of a personnel detection unit based on the YOLOX target detection algorithm, and use the sample monitoring image set and the sample personnel feature set as training data to train the personnel detection unit until convergence, and obtain a personnel detection unit that meets the expected conditions.

[0040] Among them, the crowd analysis model construction module 20 further includes: a target tracking unit construction unit, which is used to construct a target tracking unit based on the ByteTrack tracking algorithm, wherein the target tracking unit includes an input layer and an output layer, the input data of the input layer are personnel features, and the output data of the output layer are target ID and target motion trajectory, wherein each personnel feature has a uniquely identified target ID.

[0041] The specific configuration of the result acquisition module 30 will be described in detail below. As described above, the video stream data is input into the crowd flow analysis model, and the personnel detection and target tracking are performed by the personnel detection unit and the target tracking unit to obtain the personnel detection result and the target tracking result. The result acquisition module 30 further includes: a personnel detection unit, which is used to perform personnel detection on the video stream data through the personnel detection unit and output the personnel detection result; and a target tracking result output unit, which is used to transfer the video stream data and the personnel detection result to the target tracking unit if the personnel detection result is not 0, perform continuous target tracking, and output the target tracking result.

[0042] The specific configuration of the crowd flow statistics module 40 will be described in detail below. As described above, crowd flow statistics are performed according to the personnel detection results and target tracking results, and the crowd flow statistics results are visualized through a heat map to generate a regional crowd flow density distribution map. The crowd flow statistics module 40 further includes: a target area segmentation unit, the target area segmentation unit is used to segment the target area according to a predetermined division rule to obtain multiple target sub-areas, wherein each target sub-area is marked with an area; a regional crowd flow number acquisition unit, the regional crowd flow number acquisition unit is used to count the number of people in the multiple target sub-areas in the video stream data according to the personnel detection results and target tracking results, and obtain the number of people in multiple areas; a sub-area crowd flow density calculation unit, the sub-area crowd flow density calculation unit is used to calculate the crowd flow density of multiple sub-areas according to the crowd flow numbers of multiple areas and the areas of multiple areas; a visualization display unit, the visualization display unit is used to visualize the crowd flow density of the multiple sub-areas through a heat map to generate a regional crowd flow density distribution map.

[0043] The specific configuration of the flow control module 50 will be described in detail below. As described above, the crowd density distribution map of the region is identified, and the high-density region is determined to be subjected to flow control. The flow control module 50 further includes: a region screening unit, which is used to perform region screening on the crowd density distribution map of the region according to a predetermined color index to determine a plurality of high-density regions; and a flow control unit, which is used to perform flow control on the plurality of high-density regions according to a predetermined control scheme.

[0044] The real-time crowd monitoring and early warning device for a scenic spot provided by an embodiment of the present invention can execute the real-time crowd monitoring and early warning method for a scenic spot provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0045] Although the present application makes various references to certain modules in the device according to the embodiments of the present application, any number of different modules may be used and run on the user terminal and / or server, and the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0046] The above specific implementation manner does not constitute a limitation to the protection scope of the present application. It should be understood by those skilled in the art that various modifications, combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the protection scope of the present application. In some cases, the actions or steps recorded in the present application can be performed in an order different from that in the embodiment and can still achieve the desired results. In addition, the process depicted in the accompanying drawings does not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A real-time crowd flow monitoring and early warning method for a scenic spot, characterized in that: include: Perform video monitoring of the target area through an image sensor array to obtain video stream data within a predetermined time window; Constructing a crowd flow analysis model, wherein the crowd flow analysis model includes a personnel detection unit and a target tracking unit; Inputting the video stream data into the crowd flow analysis model, performing personnel detection and target tracking through the personnel detection unit and the target tracking unit, and obtaining personnel detection results and target tracking results; Perform crowd flow statistics based on the personnel detection results and target tracking results, visualize the crowd flow statistics results through a heat map, and generate a regional crowd flow density distribution map; Identify the crowd density distribution map of the area and determine the high-density area for flow control.

2. The real-time crowd flow monitoring and early warning method for scenic spots according to claim 1 is characterized in that: The image sensor array is used to cover the target area in full viewing angle, and includes N cameras, wherein each camera corresponds to a predetermined installation position, and N is an integer greater than 1.

3. The real-time crowd flow monitoring and early warning method for scenic spots according to claim 1 is characterized in that: Build a person detection unit, including: Collect a sample monitoring image set, and annotate the personnel features in different sample monitoring images to obtain a sample personnel feature set; The network structure of the personnel detection unit is constructed based on the YOLOX target detection algorithm. The sample monitoring image set and the sample personnel feature set are used as training data to train the personnel detection unit until convergence, so as to obtain a personnel detection unit that meets the expected conditions.

4. The real-time crowd flow monitoring and early warning method for scenic spots according to claim 1 is characterized in that: Build a target tracking unit, including: A target tracking unit is constructed based on the ByteTrack tracking algorithm, wherein the target tracking unit includes an input layer and an output layer, wherein the input data of the input layer are personnel features, and the output data of the output layer are target ID and target motion trajectory, wherein each personnel feature has a uniquely identified target ID.

5. The real-time crowd flow monitoring and early warning method for scenic spots according to claim 1 is characterized in that: Performing personnel detection and target tracking by the personnel detection unit and the target tracking unit to obtain personnel detection results and target tracking results includes: Performing personnel detection on the video stream data by the personnel detection unit, and outputting the personnel detection result; If the personnel detection result is not 0, the video stream data and the personnel detection result are transferred to the target tracking unit, the target is continuously tracked, and the target tracking result is output.

6. The real-time crowd flow monitoring and early warning method for scenic spots according to claim 1 is characterized in that: The crowd flow statistics are performed according to the personnel detection results and the target tracking results, and the crowd flow statistics results are visualized through a heat map to generate a regional crowd flow density distribution map, including: According to a predetermined division rule, the target area is divided to obtain a plurality of target sub-areas, wherein each target sub-area is marked with an area; According to the personnel detection result and the target tracking result, the number of people flows in the multiple target sub-areas in the video stream data is counted to obtain the number of people flows in multiple areas; Calculate the crowd density of multiple sub-regions according to the crowd numbers of the multiple regions and the areas of the multiple regions; The crowd density of the multiple sub-areas is visualized through a heat map to generate a regional crowd density distribution map.

7. The real-time crowd flow monitoring and early warning method for scenic spots according to claim 1 is characterized in that: Identify the crowd density distribution map of the area and determine the high-density area for flow control, including: Performing regional screening on the regional pedestrian density distribution map according to a predetermined color index to determine a plurality of high-density areas; According to a predetermined control scheme, current limiting control is performed on the multiple high-density areas.

8. A real-time crowd flow monitoring and early warning device for scenic spots, characterized in that: The device is used to implement the real-time crowd flow monitoring and early warning method for a scenic spot according to any one of claims 1 to 7, and the device comprises: A video monitoring module, which is used to perform video monitoring of a target area through an image sensor array and obtain video stream data within a predetermined time window; A crowd flow analysis model construction module, wherein the crowd flow analysis model construction module is used to construct a crowd flow analysis model, wherein the crowd flow analysis model includes a personnel detection unit and a target tracking unit; A result acquisition module, the result acquisition module is used to input the video stream data into the crowd flow analysis model, perform personnel detection and target tracking through the personnel detection unit and the target tracking unit, and obtain personnel detection results and target tracking results; A crowd flow statistics module, which is used to perform crowd flow statistics based on the personnel detection results and target tracking results, visualize the crowd flow statistics results through a heat map, and generate a regional crowd flow density distribution map; A flow limiting control module is used to identify the crowd density distribution map of the area and determine the high-density area for flow limiting control.

Images