Automatic customer profiling method for historical and cultural blocks based on high-precision crowd flow monitoring

Abstract

The present invention discloses a method for automatically profiling customers in a historical and cultural block based on high-precision crowd flow monitoring, which relates to the technical field of urban planning. The method comprises the following steps: inputting boundary data and longitude and latitude information of a target historical and cultural block into a cloud computing platform, collecting basic data of the target historical and cultural block, constructing a spatial database and a business database, and generating points to be inspected. Information data of the inspection points are collected through wearable intelligent devices and uploaded to the cloud computing platform. The real-time data is processed through an automatic customer profiling unit of the cloud computing platform, and information of people at the inspection points is identified based on a gender recognition module, an age recognition module, an attribute recognition module, an emotion recognition module, and a behavior recognition module. Then, through deep learning training, a crowd portrait of the historical and cultural block is realized. Data analysis, monitoring reports, and abnormal evaluation are performed on the cloud computing platform, and visual display is performed in wearable intelligent facilities, thereby outputting diagnosis results and associated corresponding control strategies.

Description

Technical Field

[0001] The present invention relates to the technical field of urban planning, and in particular to an automatic customer profiling method for a historical and cultural block based on high-precision crowd flow monitoring. Background Art

[0002] With the construction of historical and cultural blocks in my country and the continuous improvement of people's living standards, historical and cultural blocks, as important attractions in the city, attract tourists and cultural enthusiasts to visit. However, with the increase in the flow of people, the management and service of historical and cultural blocks face many challenges. First, the increased mobility of people in historical and cultural blocks requires more management personnel; second, the increase in the flow of people leads to an increase in congestion points in historical blocks, which may lead to potential stampedes and other safety hazards; third, the high-density crowds in historical and cultural blocks are complex, and customer group analysis is needed to formulate countermeasures. Therefore, how to achieve real-time identification of abnormal congestion points within historical and cultural blocks, and control them according to the behavioral characteristics and preferences of tourists, has gradually become an important link in improving the management level and service quality of historical and cultural blocks.

[0003] At present, the common methods for analyzing the customer groups in historical and cultural blocks are: one is based on the traditional manual sampling questionnaire and observation method, which manually analyzes the number, characteristics, activities and other information of tourists to analyze the composition and behavior patterns of the customer groups. However, this method consumes a lot of manpower and material resources, and has a long data collection cycle, high cost, small sample, and it is difficult to obtain customer group information in a timely and accurate manner. The real-time performance is poor, and it is difficult to optimize the management and operation of historical and cultural blocks. The other is data collection and analysis based on mobile Internet and intelligent collection, which collects the location information, movement trajectory, length of stay and other information of mobile tourists through mobile phone APP, WIFI signal and other facilities, so as to realize the monitoring of the customer group. This method is convenient to obtain data, has strong real-time performance, and can warn of congestion points, but it is difficult to effectively identify the attributes of the customer group. At the same time, in terms of data collection, tourists need to carry smart phones, there is a certain recognition gap in the elderly and children, and it is difficult to achieve targeted control in block management. Summary of the invention

[0004] In order to solve the deficiencies mentioned in the above background technology, the purpose of the present invention is to provide a method for automatically profiling the customer groups in historical and cultural blocks based on high-precision crowd flow monitoring.

[0005] The purpose of the present invention can be achieved by the following technical solution: A method for automatically profiling customers in historical and cultural blocks based on high-precision crowd flow monitoring, comprising the following steps:

[0006] Step 1: Input the boundary data and longitude and latitude information of the target historical and cultural block into the cloud computing platform equipped with Windows 10 system, and the data of the target historical and cultural block will be automatically collected to build the business database and space database. The cloud computing platform will automatically generate the inspection points of the historical and cultural block to be inspected, and after the inspection points are generated, they will be automatically transmitted to the human-computer interactive display screen of the backpack information collection device.

[0007] Furthermore, the method for automatically constructing the spatial database of the historical blocks to be inspected is that the cloud computing platform automatically downloads the map data file with the suffix .osm through the OSM map interface based on the boundary data and longitude and latitude information of the historical and cultural blocks to be inspected, which includes the traffic network nodes, traffic route data, river data, and land boundary data in the target area. The cloud computing platform automatically identifies the green spaces, squares, roads and building areas of the historical and cultural blocks to be inspected through the built-in image recognition method, and stores the above data in .shp format as a spatial database.

[0008] Furthermore, the method of automatically constructing the business type database of the historical blocks to be inspected is that the cloud computing platform automatically obtains the business type POI (point of interest) data and historical and cultural resource point data of the historical and cultural blocks to be inspected based on the open API interface of Gaode Map, matches them to the WGS-84 coordinate system, and stores the above data in .shp format as a business type database.

[0009] Furthermore, the automatic generation method of the inspection points of the historical and cultural blocks to be inspected is that the cloud computing platform automatically divides the historical and cultural blocks to be inspected into grids based on the input historical and cultural boundary data, and generates control points with a spacing of 10m along the roads based on the road data in the spatial database, and uses the control points as the inspection points of the historical block.

[0010] Step 2: The staff will carry a backpack-type information collection device and Moverio BT-40 AR glasses to conduct inspections at the inspection points marked on the human-computer interactive display screen. The backpack-type information collection device will automatically collect and upload information at the inspection points, including passenger flow density data, 360° panoramic photo data, time data, geographic location data, and the inspection status and inspection time of each inspection point. After receiving the uploaded information data, the cloud computing platform will automatically verify it based on the information data collection standards. If the information verification fails, the cloud computing platform will feedback the inspection point where the information verification failed to the human-computer interactive display screen to guide the staff to conduct a second collection.

[0011] Furthermore, the inspection point information of the historical and cultural block is automatically collected and uploaded, that is, the staff carries a backpack information collection device equipped with a WIFI probe, a 360° panoramic camera, a GPS positioning system and a human-computer interaction display screen, and a Moverio BT-40 model AR glasses, and reaches the designated inspection points through the human-computer interaction display screen for inspection. The duration of stay at each point is 1 minute. During the inspection, the staff clicks the start inspection button of this inspection point on the human-computer interaction display, and the backpack information collection device will start the information collection work for 1 minute. Through the crowd density monitoring unit and the scene information collection unit, combined with the WIFI probe, 360° panoramic camera, and GPS positioning system, the passenger flow density data, 360° panoramic photo data, time data, and geographic location data of the inspection point are automatically collected. After the inspection point information collection is completed, the backpack information collection device will automatically record the inspection status and inspection time of this inspection point, and upload the collected information data to the cloud computing platform through the information collection and sending unit. Among them, the crowd density monitoring unit is used to record the latitude and longitude information of all people collected by the WIFI probe with the backpack information collection device as the center and a radius of 500m, and record the corresponding time, as well as the crowd density with the inspection point as the center and a radius of 500m, and store it as a file in the format of .xlsx according to the naming method of "collection time-collection inspection point"; the scene information collection unit is used to record the panoramic photo information taken by the backpack information collection device at each inspection point for 1 minute through the 360° panoramic camera per second, and record the corresponding time and longitude and latitude information, and store it as a .jpg file according to the naming method of "collection time-collection inspection point"; the collected information sending unit is used to send all information files collected by the crowd density monitoring unit and the scene information collection unit to the cloud computing platform.

[0012] Furthermore, the cloud computing platform automatically verifies information in a method that the cloud computing platform verifies information based on the information data collection standard. If the information verification fails, the cloud computing platform will feedback the inspection point where the information verification fails to the human-computer interaction display screen to guide the staff to conduct a secondary collection. The information data collection standard refers to the occlusion rate of objects in the photos taken by the panoramic camera being less than 30%.

[0013] Step 3: The cloud computing platform has a built-in customer group automatic profiling unit, which is linked to the information collection sending unit of the backpack information collection device through the 5G network, receives passenger flow density data, 360° panoramic photo data, time data, and geographic location data, and performs automatic profiling. The customer group automatic profiling unit pre-processes the passenger flow density data and 360° panoramic photo data through the built-in DGX computer workstation, automatically builds a summary database of the crowd information of the historical and cultural blocks, and imports it into the open map platform, and integrates it into the historical and cultural blocks sand table after unifying the coordinate system. The workstation builds a summary database of the crowd information of the historical and cultural blocks and numbers the crowd IDs, and inserts the gender recognition module, age recognition module, attribute recognition module, emotion recognition module, and behavior recognition module into the unit. The workstation uses deep learning training to build a convolutional neural network model, automatically calculates the individual characteristics of each ID, and identifies the basic characteristics and spatiotemporal behavior characteristics of the mobile population in the historical and cultural blocks, where the basic characteristics include gender, age, attributes, and emotions, and the spatiotemporal behavior characteristics include behavior recognition, mobility preference, and residence preference. The workstation fuses and matches the data after feature calculation, integrates different feature data of the same individual, and identifies and extracts the crowd portraits of historical and cultural blocks. Based on the crowd features identified by the customer group automatic portrait unit, the workstation clusters and identifies them through spatial clustering (ST-DBSCAN) technology, XGBoost tree algorithm and hidden Markov model, extracting 3 types of age, emotion, and attribute recognition filters, a total of 32 types of crowd portraits, of which age includes three levels: children, teenagers, middle-aged people, and the elderly; attributes include local residents and foreign tourists; and emotions include four levels: joy, anger, sadness, and fear.

[0014] Furthermore, the method for preprocessing the passenger flow density data and 360° panoramic photo data is that the automatic customer portrait unit has a built-in DGX computer workstation, which constructs a crowd image data set from the 360° panoramic photo data, performs image grayscale processing, image filtering, edge detection and binarization processing, unifies the image size and numbers it, and connects the time data and geographic location data with the crowd passenger flow density data and crowd image data.

[0015] Furthermore, the construction of a summary database of crowd information in historical and cultural blocks and crowd ID numbering, that is, the automatic customer portrait unit has a built-in DGX computer workstation, unifies the coordinate information of the crowd passenger flow density data set and the crowd image data set with coordinate information into a unified spatial coordinate system, and converts the data information except the image information and coordinate information into recognizable text information, and stores it in CSV file format after dimensionless processing; the crowd numbering is to obtain the scene image screen of each inspection point in the crowd image data set, extract and identify the face image in the scene image screen, and determine the temporary ID and location attributes of the face image; the basic characteristics and spatiotemporal behavior characteristics of the mobile crowd in the historical and cultural blocks are identified in the step three, and the automatic customer portrait unit of the historical and cultural blocks is constructed based on the gender recognition module, age recognition module, attribute recognition module, emotion recognition module, and behavior recognition module of the automatic customer portrait unit built into the cloud computing platform to automatically portrait the historical and cultural blocks.

[0016] Furthermore, the gender recognition module, age recognition module, emotion recognition module, and behavior recognition module are embedded with facial recognition, expression analysis, action recognition functions and classifiers in the cloud platform, and 10 areas where micro-expression muscle movements occur frequently and the human head and limb areas are located. Human photos are classified according to human facial states, expressions and actions, and gender, age, emotion, and behavior original data sets are obtained respectively. After vectorization processing, they are used as machine learning labels and divided into training set: verification set: test set = 6:2:2. A supervised classification machine learning algorithm is used to perform machine learning training, generate automatic recognition model clusters, and construct recognition modules; belonging to The individual recognition module extracts individual images from the scene image screen, identifies whether they have large carry-on luggage, and determines whether they are local residents or foreign tourists; the flow preference, including two indicators of flow direction and flow speed, identifies the inspection points where individuals stay according to the scene image screen in the crowd image data set, connects the inspection points in chronological order, determines the individual's flow direction and calculates the flow speed; the residence preference, the total residence time refers to the individual's cumulative residence time in a day, records the time points when individuals enter and leave the inspection point range according to the scene image screen in the crowd image data set, forms a data set, and records the number of times they stay within the inspection point range. The total residence time calculation formula is, ,in, represents the total length of time that individual m stays, i represents the number of times individual m stays within the inspection point range, It is represented as the time point when individual m leaves the inspection point for the i-th time. It is represented as the time point when individual m enters the inspection point range for the i-th time.

[0017] Step 4: Based on the passenger density data collected by the crowd density monitoring unit during the inspection of wearable smart devices, the cloud computing platform automatically uses the customer monitoring unit to perform data analysis in combination with the customer portrait results: based on the automatic customer portrait described in step 3, complete the real-time customer data processing; calculate the overall passenger density of the inspection points and the passenger density of each customer portrait based on the real-time passenger flow data; set the passenger density threshold of each inspection point in the historical and cultural blocks, and mark the abnormal inspection points of the customer group according to the passenger density of the inspection points; add and divide abnormal inspection points based on the density of each customer portrait at each inspection point. The cloud computing platform outputs monitoring reports and abnormal evaluations based on the standardized data analysis of the customer monitoring unit.

[0018] Furthermore, the method for processing real-time customer data based on automatic customer portrait is to automatically match the real-time customer data with the automatic customer portrait features (including age, emotion, and attributes) based on the ID, and sort out the real-time customer flow data according to each inspection point. The data information includes timestamp, ID, inspection point, age, emotion, and attributes.

[0019] Furthermore, the method of calculating the passenger flow density of the whole inspection point and each customer group portrait according to the real-time passenger flow data is to automatically perform statistical calculations on the cloud computing platform based on the crowd density unit collection results of each inspection point, obtain the average passenger flow of the whole and each customer group portrait, and automatically generate the passenger flow density of different customer group portraits. The specific formula is: passenger flow density of customer group k , s is the inspection point area, t is the fixed time, U k is the number of IDs passed by customer group k in a fixed time period.

[0020] Furthermore, the method of setting the passenger flow density threshold of each inspection point in the historical and cultural block, and marking the abnormal inspection points of the customer group according to the passenger flow density of the inspection points, is to comprehensively set the passenger flow density threshold of each customer group portrait based on the location characteristics and venue capacity, safety and fire protection factors, and service support. In terms of location characteristics and venue capacity, the maximum safe capacity is determined based on the available space area and entrance and exit space layout of the inspection point in the spatial database, and 75% of the maximum safe capacity is used as the upper limit threshold of the passenger flow density. In terms of safety and fire protection factors, the maximum number of people accommodated is calculated based on local fire protection regulations and standards, and the safe passenger flow density threshold is set accordingly. In terms of service support, the maximum passenger flow received is evaluated based on the number and type of POIs in the business database, and the maximum passenger flow density threshold is set accordingly. The passenger flow density threshold is the minimum value of 75% of the maximum safe capacity, the safe passenger flow density threshold, and the maximum passenger flow density threshold. Each inspection point is marked as an abnormal inspection point according to its overall customer density and the corresponding customer density threshold: when the passenger flow density is higher than the passenger flow density threshold, the point is marked as an abnormal customer inspection point.

[0021] Furthermore, the method for adding and dividing abnormal inspection points based on the density of each customer group portrait at each inspection point is to set the customer density threshold at each inspection point considering the portrait characteristics of vulnerable groups (children, the elderly) to ensure the safety of vulnerable groups. When the density of each customer group is higher than the passenger flow density threshold of the corresponding customer group, the point is added and marked as an abnormal inspection point for the customer group (see the table below). Each abnormal inspection point is divided into abnormal inspection points according to the passenger flow density corresponding to the 32 customer groups described in step three, and the customer group with the largest proportion is marked.

[0022] Furthermore, the method of outputting monitoring reports and abnormal evaluation reports based on the standardized data analysis of the customer group monitoring unit is to generate monitoring reports and abnormal evaluation reports based on the data analysis results of the cloud platform, and the data analysis results include: the location, type, customer group density, basic information of the number of portraits of each customer group, and customer group analysis reports. Among them, the customer group analysis report includes passenger flow density, passenger flow peak time, preferred area and customer group activity pattern.

[0023] Step 5: Combine the customer monitoring report output and abnormal evaluation results, automatically link them to the strategy library in the cloud computing platform, generate a management optimization report based on the automatic portrait of the customer group in the historical and cultural block, and feed it back to the human-computer interactive display screen. In the human-computer interactive display screen, the strategy library is combined with feedback information and real-time collected passenger flow data to transform into visual information to assist staff in patrol and management work.

[0024] Furthermore, the method of automatically associating with the policy library in the cloud computing platform, that is, combining the customer detection report output and the abnormal evaluation results of step four, automatically associates with the cloud computing platform policy library for different types of abnormal inspection points, and the associated information is the type of abnormal inspection point and the corresponding customer flow density value (see the table below).

[0025] Furthermore, the strategy library is combined with feedback information and real-time collected passenger flow data to transform into a visualization information method, that is, the passenger flow data of the surrounding environment is collected by wearable smart devices, and the analysis results of the cloud computing platform, real-time passenger flow data and related databases are fed back to the human-computer interaction display screen and the Moverio BT-40 model AR glasses in combination with GPS positioning. The passenger flow data is combined with the data analysis results, converted into visualization information in real time, and projected directly onto the display screen of the glasses, so that the staff can intuitively see the current passenger flow density, analysis results and corresponding strategies of each customer group. The system will automatically match the cloud computing analysis results, including portraits of different customer groups and their density in specific areas and abnormal customer groups. The area will be circled and highlighted with a pink icon in the augmented reality glasses, and the passenger flow density information of each customer group and the relevant location map will be matched nearby, and the real-time strategy corresponding to the strategy library will be displayed.

[0026] Beneficial effects of the present invention:

[0027] 1. Process efficiency: The present invention realizes the full-process intelligent operation of crowd flow monitoring and automatic allocation strategy in historical and cultural blocks from data collection, data transmission, automatic profiling, crowd monitoring, association strategy, information feedback to visual interaction through a backpack information collection device equipped with a WIFI probe, a 360° panoramic camera, a GPS positioning system and a human-computer interactive display screen, a cloud processing terminal equipped with a Windows 10 system, a human-computer interactive display screen, and Moverio BT-40 AR glasses. This equipment improves the scientific nature of traditional crowd flow monitoring, and automatically profiles the crowd. Starting from the preferences of different portraits of the crowd, it improves the work efficiency and accuracy of crowd flow monitoring and management in historical and cultural blocks, effectively reduces time and labor costs, and improves the real-time and accuracy of crowd flow monitoring and guidance.

[0028] 2. Objectivity of feedback: The present invention ensures the effectiveness of the generation of crowd guidance strategies in the block through a cloud computing platform that covers dynamic monitoring of customer flow, abnormal evaluation and early warning, and automatic feedback of the associated strategy library, breaking through the limitations of human judgment in the traditional crowd guidance process and avoiding the uncontrollability of the effectiveness of previous crowd evacuation plans. After setting the passenger flow density threshold of each inspection point, the cloud computing platform can be used to complete dynamic monitoring of customer flow, abnormal evaluation and early warning, and automatic feedback of the associated strategy library, thereby improving the reliability and efficiency of the customer flow guidance plan.

[0029] 3. Interactive visualization: The present invention provides block managers with intuitive, easy-to-understand, convenient and efficient presentation of historical and cultural block-scale customer portraits and dynamic monitoring results of human flow through human-computer interactive display screens and Moverio BT-40 model AR glasses, and realizes interactive decision-making of customer flow monitoring and automatic allocation strategies in historical and cultural blocks, thus improving business scenarios. The present invention adopts an interactive mode combining human-computer interactive display screens and Moverio BT-40 model AR glasses, providing two types of interactive results: two-dimensional image type and three-dimensional dynamic type. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A flow chart of the method implemented by the present invention;

[0031] Figure 2 This is a diagram showing the composition of the wearable device module in the present invention;

[0032] Figure 3 is a schematic diagram of the structure of the wearable device in the present invention;

[0033] Figure 4 This is a flow chart of the method for automatically profiling the customer base in a historical district of the present invention. DETAILED DESCRIPTION

[0034] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. Figure 1 The flowchart of the method for implementing the present invention is obvious. The described embodiment is only a part of the embodiment of the present invention, not all the embodiments. Based on the embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] An automatic customer profiling method for historical and cultural blocks based on high-precision crowd monitoring requires the use of wearable devices and cloud computing platforms, such as Figure 2 The wearable device includes a backpack-type information collection device and a Moverio BT-40 model AR glasses, as shown in FIG. Figure 3 As shown, the backpack information collection device is a backpack type, equipped with a WIFI probe, a 360° panoramic camera, a GPS positioning system and a human-computer interactive display screen, and has a crowd density monitoring unit, a scene information collection unit, and a collection information sending unit. The cloud computing platform is equipped with a cloud processing terminal and a human-computer interactive display screen of Windows 10 system, and has a customer group automatic portrait unit, a customer group monitoring unit and a customer group abnormality evaluation unit, which are used to receive backpack information collection results, store data, calculate data and output data.

[0036] The technical solution of the present invention is described in detail with the Confucius Temple historical and cultural district in Nanjing, Jiangsu Province:

[0037] (1) Staff input data and generate inspection points for the Confucius Temple Historical and Cultural Block in Nanjing on the cloud computing platform, including:

[0038] (1.1) The staff inputs the boundary data and latitude and longitude information of the Confucius Temple Historical and Cultural Block in Nanjing into the cloud computing platform. The cloud computing platform automatically downloads the map data file with the suffix .osm based on the OSM map interface, which contains the traffic network nodes, traffic route data, river data, and land boundary data in the Confucius Temple Historical and Cultural Block in Nanjing. The cloud computing platform automatically identifies the green space, square, road, and building area of ​​the Confucius Temple Historical and Cultural Block in Nanjing through the built-in image recognition method, and stores it as a spatial database. At the same time, the cloud computing platform automatically obtains the business POI (point of interest) data and historical and cultural resource point data of the historical and cultural block to be inspected based on the open API interface of AutoNavi Maps, matches it to the WGS-84 coordinate system, and stores it as a business database.

[0039] (1.2) The cloud computing platform automatically divides the Nanjing Confucius Temple Historical and Cultural Block into a grid based on the input historical and cultural boundary data, and generates inspection points at 10m intervals based on the roads. After the inspection points are generated, they are automatically transmitted to the human-computer interactive display screen of the backpack information collection device. The inspection points refer to the grid division of the historical and cultural block to be inspected by the cloud computing platform, and the generation of control points at 10m intervals based on the roads as the inspection points of the historical block.

[0040] (2) Inspection, information collection and information upload of the Confucius Temple Historical and Cultural District in Nanjing were carried out through backpack-type wearable smart devices, including:

[0041] (2.1) Staff members of the Confucius Temple Historical and Cultural District carry backpack-type information collection devices and use the human-computer interactive display screen to reach the marked inspection points for inspection. The duration of stay at each point is 1 minute.

[0042] (2.2) During the inspection, the staff clicks the start inspection button of the inspection point through the human-computer interaction display, and the backpack information collection device will start the information collection work for 1 minute. The crowd density monitoring unit records the number of people within a radius of 500m with the backpack information collection device as the center, and records the corresponding time and longitude and latitude information. The scene information collection unit records the panoramic photo information taken by the backpack information collection device when it is in operation, and records the corresponding time and longitude and latitude information. Combined with the WIFI probe, 360° panoramic camera, and GPS positioning system, the passenger density data, 360° panoramic photo data, time data, and geographic location data of the inspection point are automatically collected.

[0043] (2.3) After the inspection point information is collected, the backpack information collection device will automatically record the inspection status and inspection time of the inspection point, and automatically upload the information collected by the crowd density monitoring unit and the scene information collection unit to the cloud computing platform through the information collection sending unit.

[0044] (2.4) After receiving the information data, the cloud computing platform will automatically check based on the information data collection standards to determine whether the object occlusion rate in the photos taken by the panoramic camera is less than 30%. If the information verification fails, the cloud computing platform will feedback the inspection point where the information verification failed to the human-computer interactive display screen, and guide the staff to conduct a second collection of point data for the Nanjing Confucius Temple Historical and Cultural District.

[0045] (3) The cloud computing platform automatically profiles the customer base of the Confucius Temple Historical and Cultural District in Nanjing, such as Figure 4 As shown, specifically including:

[0046] (3.1) The cloud computing platform has a built-in automatic customer profiling unit, which is connected to the information collection and sending unit of the backpack-type information collection device through the 5G network, receives passenger flow density data, 360° panoramic photo data, time data, and geographic location data of the Confucius Temple Historical and Cultural District in Nanjing, and performs automatic profiling.

[0047] (3.2) The automatic customer profiling unit has a built-in DGX computer workstation, which constructs a crowd image dataset from 360° panoramic photo data, performs image grayscale processing, image filtering, edge detection and binarization processing, unifies the image size and numbers it, connects the time data and geographic location data with the crowd passenger flow density data and crowd image data, automatically constructs a summary database of crowd information in historical and cultural blocks and imports it into the open map platform. After unifying the coordinate system, it is embedded in the Nanjing Confucius Temple Historical and Cultural Block Sand Table.

[0048] (3.3) The workstation constructs a crowd information summary database for the Confucius Temple Historical and Cultural District in Nanjing, unifies the coordinate information of the crowd passenger flow density dataset and the crowd image dataset with coordinate information into a unified spatial coordinate system, and converts data information other than image information and coordinate information into recognizable text information, and stores it in CSV file format after dimensionless processing; the crowd numbering is to obtain the scene image screen of each inspection point in the crowd image data set, extract and identify the face image in the scene image screen, and determine the temporary ID and location attributes of the face image.

[0049] (3.4) The workstation places the temporary ID, facial image and location attributes of the people in Nanjing Confucius Temple Historical and Cultural District into the gender recognition module, age recognition module, attribute recognition module, emotion recognition module and behavior recognition module in the unit, and constructs a convolutional neural network model through deep learning training. It automatically calculates the individual characteristics of each ID and identifies the basic characteristics and spatiotemporal behavior characteristics of the mobile population in Nanjing Confucius Temple Historical and Cultural District.

[0050] (3.5) Relying on the gender recognition module, age recognition module, emotion recognition module, and behavior recognition module of the built-in customer automatic profiling unit of the cloud computing platform, the cloud platform is equipped with facial recognition, expression analysis, action recognition functions and classifiers, and 10 areas where micro-expression muscle movements occur frequently and the human head and limb areas are located. Human photos are classified according to human facial states, expressions and actions, and the original data sets of gender, age, emotion and behavior are obtained respectively. After vectorization processing, they are used as machine learning labels and divided into training set: validation set: test set = 6:2:2. A supervised classification machine learning algorithm is used to conduct machine learning training, generate automatic recognition model clusters, and construct a recognition module.

[0051] (3.6) Relying on the attribute recognition module of the customer automatic profiling unit built into the cloud computing platform, individual images in the scene image are extracted to identify whether the individual has large carry-on luggage and determine whether the individual is a local resident or a foreign tourist.

[0052] (3.7) Based on the scene images in the crowd image dataset, identify the inspection points where the individual resides, connect the inspection points in chronological order, and determine the individual's flow preference, including two indicators: flow direction and flow speed.

[0053] (3.8) Based on the scene images in the crowd image dataset, the time points when individuals enter and leave the inspection point range are recorded to form a dataset, and the number of times they stay within the inspection point range is recorded to determine the individual's residence preference. The total residence time refers to the cumulative residence time of an individual in a day. The total residence time calculation formula is, .

[0054] (3.9) Among them, represents the total length of time that individual m stays, i represents the number of times individual m stays within the inspection point range, It is represented as the time point when individual m leaves the inspection point for the i-th time. It is represented as the time point when individual m enters the inspection point range for the i-th time.

[0055] (3.10) The workstation fuses and matches the data after feature calculation, integrates different feature data of the same individual, and identifies and extracts the population portrait of Nanjing Confucius Temple Historical and Cultural District.

[0056] (3.11) Based on the crowd characteristics identified by the automatic customer portrait unit, the workstation clusters and identifies them through spatial clustering (ST-DBSCAN) technology, XGBoost tree algorithm and hidden Markov model, and extracts three types of feature recognition filters: age, emotion and attribute. A total of 32 types of crowd portraits of Nanjing Confucius Temple Historical and Cultural District are obtained, among which the age includes three levels: children, teenagers, middle-aged people and the elderly; the attributes include local residents and foreign tourists; and the emotions include four levels: joy, anger, sadness and fear.

[0057] (4) The cloud computing platform automatically outputs customer monitoring reports and conducts abnormality evaluation:

[0058] (4.1) The cloud computing platform automatically processes the real-time customer data collected by wearable smart devices. The customer profile features (including age, emotion, and attributes) are automatically matched to the real-time customer data based on the ID, and the real-time customer flow data is sorted according to each inspection point. The data information includes timestamp, ID, inspection point, age, emotion, and attributes.

[0059] (4.2) The cloud computing platform automatically calculates the overall passenger flow density of each customer group at the inspection point. The results of the crowd density unit collection at each inspection point are automatically statistically calculated in the cloud computing platform to obtain the average passenger flow of the whole and each customer group portrait, and automatically generate the passenger flow density of different customer group portraits.

[0060] (4.3) The cloud computing platform automatically sets the passenger density threshold of the inspection points and marks the abnormal inspection points according to the passenger density. The passenger density threshold of each customer group portrait is set based on the location characteristics and venue capacity, safety and fire protection factors, and service support. Each inspection point is marked as an abnormal inspection point according to its overall customer density and the corresponding customer density threshold: when the passenger density is higher than the passenger density threshold, the point is marked as an abnormal inspection point for the customer group.

[0061] (4.4) The cloud computing platform automatically adds and divides abnormal inspection points based on the density of each customer group portrait at each inspection point. Each inspection point sets the customer density threshold considering the portrait characteristics of vulnerable groups to ensure the safety of vulnerable groups. When the density of each customer group is higher than the corresponding customer group passenger flow density threshold, the point is added and marked as an abnormal customer group inspection point.

[0062] Each abnormal inspection point is divided into abnormal inspection points according to the corresponding passenger flow density of the 32 customer groups described in step 3, and the customer group with the largest proportion is marked. Based on the above calculation, the cloud computing platform outputs monitoring reports and abnormal evaluations based on the standardized data analysis of the customer group monitoring unit.

[0063] (5) Generate management optimization reports based on automatic customer profiles of historical and cultural blocks

[0064] Combined with the customer monitoring report output and abnormal evaluation results, it is automatically associated with the strategy library in the cloud computing platform to generate a management optimization report based on the automatic portrait of the customer group in the historical and cultural blocks, and fed back to the human-computer interactive display screen. In the human-computer interactive display screen, the strategy library is combined with feedback information and real-time collected passenger flow data to transform into visual information. Wearable devices assist staff in patrol and management work based on interactive feedback and flexible configuration.

[0065] (5.1) The cloud computing platform automatically associates the customer monitoring report output and abnormal evaluation results with the strategy library and feeds them back to the human-computer interactive display screen. The associated information is the type of abnormal inspection point and the corresponding customer flow density value.

[0066] (5.2) Wearable smart devices automatically combine the strategy library with feedback information and real-time collected passenger flow data and convert them into visual information. Combined with GPS positioning, the visual information is fed back to the human-computer interaction display screen and the Moverio BT-40 model AR glasses, and projected directly onto the display screen of the glasses, so that the staff can intuitively see the current passenger flow density, analysis results and corresponding strategies of each customer group. The system will automatically match the cloud computing analysis results, including different customer group portraits and their density in specific areas and abnormal customer groups. The area will be circled and highlighted in the augmented reality glasses with a pink icon, and the relevant passenger flow density information of each customer group, relevant location maps, and real-time strategies corresponding to the strategy library will be matched nearby. The staff can query the specific information of a larger area or customer group and receive early warning information in real time through voice commands or gesture control, and carry out daily inspections of the Confucius Temple Historical and Cultural District based on real-time data updates.

Claims

1. A method for automatically profiling customers in historical and cultural blocks based on high-precision crowd flow monitoring, characterized in that: The following steps are involved: Step 1: Input historical and cultural block data and generate inspection points Input the boundary data and longitude and latitude information of the target historical and cultural block into the cloud computing platform equipped with Windows 10 system, and the data of the target historical and cultural block will be automatically collected to build the business database and space database; the cloud computing platform will automatically generate the inspection points of the historical and cultural block to be inspected, and after the inspection points are generated, they will be automatically transmitted to the human-computer interaction display screen of the backpack information collection device; Step 2: Inspection of historical and cultural blocks, information collection, information upload and information verification The staff will carry a backpack-type information collection device and Moverio BT-40 AR glasses to conduct inspections at the inspection points marked on the human-computer interactive display screen. The backpack-type information collection device will automatically collect and upload information at the inspection points, including passenger flow density data, 360° panoramic photo data, time data, geographic location data, and inspection status and inspection time of each inspection point. After receiving the uploaded information data, the cloud computing platform will automatically verify it based on the information data collection standards. If the information verification fails, the cloud computing platform will feedback the inspection points where the information verification fails to the human-computer interaction display screen to guide the staff to conduct a second collection; Step 3: Automatically profile the customer base in historical and cultural districts The cloud computing platform has a built-in automatic customer profiling unit, which is linked to the information collection sending unit of the backpack-type information collection device through the 5G network, receives passenger flow density data, 360° panoramic photo data, time data, geographic location data, and performs automatic profiling. The automatic customer profiling unit pre-processes the passenger flow density data and 360° panoramic photo data through the built-in DGX computer workstation, automatically builds a summary database of the crowd information of the historical and cultural blocks, imports it into the open map platform, and integrates it into the historical and cultural blocks sand table after unifying the coordinate system. The workstation builds a summary database of the crowd information of the historical and cultural blocks and numbers the crowd IDs, and inserts the gender recognition module, age recognition module, attribute recognition module, emotion recognition module, and behavior recognition module into the unit. The workstation uses deep learning training to build a convolutional neural network model, automatically calculates the individual characteristics of each ID, and identifies the basic characteristics and spatiotemporal behavior characteristics of the mobile crowd in the historical and cultural blocks, where the basic characteristics include gender, age, attributes, and emotions, and the spatiotemporal behavior characteristics include behavior recognition, mobility preference, and residence preference. The workstation fuses and matches the data after feature calculation, integrates different feature data of the same individual, identifies and extracts the crowd portraits of historical and cultural blocks; based on the crowd features identified by the customer group automatic portrait unit, the workstation clusters and identifies them through spatial clustering technology, XGBoost tree algorithm and hidden Markov model, extracts 3 types of age, emotion, and attribute recognition filters, and a total of 32 types of crowd portraits, among which the age includes three levels of children, teenagers, middle-aged people, and the elderly; the attributes include local residents and foreign tourists; and the emotions include four levels of joy, anger, sadness, and fear; Step 4: Cloud computing platform customer monitoring report output and abnormal evaluation Based on the passenger flow density data collected by the crowd density monitoring unit during the inspection of wearable smart devices, the cloud computing platform automatically uses the customer group monitoring unit to perform data analysis in combination with the customer group portrait results: based on the automatic customer group portrait described in step 3, complete the real-time customer group data processing; calculate the overall passenger flow density of the inspection point and each customer group portrait based on the real-time passenger flow data; set the passenger flow density threshold of each inspection point in the historical and cultural blocks, and mark the abnormal customer group inspection points according to the passenger flow density of the inspection points; Based on the density of customer profiles at each inspection point, abnormal inspection points are added and divided; the cloud computing platform outputs monitoring reports and abnormal evaluations based on standardized data analysis of customer monitoring units; Step 5: Generate a management optimization report based on the automatic portrait of the customer base in the historical and cultural blocks Combined with the customer monitoring report output and abnormal evaluation results in step 4, they are automatically associated with the strategy library in the cloud computing platform to generate a management optimization report based on the automatic portrait of the customer groups in the historical and cultural blocks, and fed back to the human-computer interactive display screen; in the human-computer interactive display screen, the strategy library is combined with the feedback information and the real-time collected passenger flow data to be converted into visual information to assist the staff in patrol and management work; In the step 3, the gender recognition module, age recognition module, emotion recognition module, and behavior recognition module are embedded with facial recognition, expression analysis, action recognition functions, and classifiers on the cloud platform, and 10 areas where micro-expression muscle movements occur frequently and the human head and limb areas are located. Human photos are classified according to human facial states, expressions, and actions, and the original data sets of gender, age, emotion, and behavior are obtained respectively. After vectorization processing, they are used as machine learning labels and divided into training set: validation set: test set = 6:2:

2. A supervised classification machine learning algorithm is used to perform machine learning training, generate an automatic recognition model cluster, and construct a recognition module; The attribute recognition module extracts individual images from the scene image, identifies whether they have large carry-on luggage, and determines whether they are local residents or foreign tourists; the flow preference, including two indicators of flow direction and flow speed, identifies the inspection points where individuals stay according to the scene image in the crowd image data set, connects the inspection points in chronological order, determines the flow direction of the individual and calculates the flow speed; the residence preference, the total residence time refers to the accumulated residence time of the individual in a day, records the time points when the individual enters and leaves the inspection point range according to the scene image in the crowd image data set, forms a data set, and records the number of times the individual stays within the inspection point range. The total residence time calculation formula is, Among them, T m represents the total length of time that individual m stays, i represents the number of times individual m stays within the inspection point range, It is represented as the time point when individual m leaves the inspection point for the i-th time. It is represented as the time point when individual m enters the inspection point range for the i-th time; The method for processing real-time customer data based on automatic customer portrait in step 4 is to automatically match the real-time customer data with the automatic customer portrait feature in step 3 based on the ID, and sort out the real-time customer flow data according to each inspection point, and the data information includes timestamp, ID, inspection point, age, emotion, and attributes; The fourth step is to calculate the passenger flow density of the inspection point as a whole and each customer group portrait according to the real-time passenger flow data, that is, based on the crowd density unit collection results of each inspection point, automatically perform statistical calculations in the cloud computing platform to obtain the average passenger flow of the whole and each customer group portrait, and automatically generate the passenger flow density of different customer group portraits. The specific formula is: passenger flow density of customer group k s is the inspection point area, t is the fixed time, U k is the number of IDs passed by customer group k in a fixed time period; The fourth step is to set the passenger flow density threshold of each inspection point in the historical and cultural block, and mark the abnormal inspection points of the customer group according to the passenger flow density of the inspection points, that is, to comprehensively set the passenger flow density threshold of each customer group portrait based on the location characteristics and venue capacity, safety and fire protection factors, and service support; in terms of location characteristics and venue capacity, the maximum safe capacity is determined based on the available space area and entrance and exit space layout of the inspection point in the spatial database, and 75% of the maximum safe capacity is used as the upper limit threshold of the passenger flow density; in terms of safety and fire protection factors, the maximum number of people accommodated is calculated based on local fire protection regulations and standards, and the safe passenger flow density threshold is set accordingly; in terms of service support, the maximum passenger flow received is evaluated based on the number and type of POIs in the business database, and the maximum passenger flow density threshold is set accordingly; the passenger flow density threshold is the minimum value of the maximum safe capacity of 75%, the safe passenger flow density threshold and the maximum passenger flow density threshold; each inspection point is marked as an abnormal inspection point according to its overall customer group density and the corresponding customer group density threshold: when the passenger flow density is higher than the passenger flow density threshold, the point is marked as an abnormal inspection point for the customer group; The fourth step is a method for adding and dividing abnormal inspection points based on the density of each customer group portrait at each inspection point, that is, the customer density threshold is set at each inspection point considering the portrait characteristics of vulnerable groups to ensure the safety of vulnerable groups; when the density of each customer group is higher than the passenger flow density threshold of the corresponding customer group, the point is added and marked as an abnormal inspection point for the customer group; each abnormal inspection point is divided into abnormal inspection points according to the passenger flow density corresponding to the 32 customer groups described in step three, and the customer group with the largest proportion is marked; The step 4 is a method for outputting monitoring reports and abnormal evaluation reports based on the standardized data analysis of the customer monitoring unit, that is, generating monitoring reports and abnormal evaluation reports according to the data analysis results of the cloud platform, and the data analysis results include: the location, type, customer density, basic information of the number of portraits of each customer group and a customer analysis report of abnormal inspection points; wherein the customer analysis report includes passenger flow density, passenger flow peak hours, preferred areas and customer activity patterns.

2. The method for automatically profiling customers in historical and cultural blocks based on high-precision crowd flow monitoring according to claim 1 is characterized in that: The method for automatically constructing the spatial database of the historical blocks to be inspected in the step 1 is that the cloud computing platform automatically downloads the map data file with the suffix .osm through the OSM map interface based on the boundary data and longitude and latitude information of the historical and cultural blocks to be inspected, which includes the traffic network nodes, traffic route data, river data, and land boundary data in the target area. The cloud computing platform automatically identifies the green space, square, road and building area of ​​the historical and cultural blocks to be inspected through the built-in image recognition method, and stores the above data in .shp format as a spatial database; the method for automatically constructing the business database of the historical blocks to be inspected in the step 1 is that the cloud computing platform automatically obtains the business POI data and historical and cultural resource point data of the historical and cultural blocks to be inspected based on the open API interface of Amap, matches them to the WGS-84 coordinate system, and stores the above data in .shp format as a business database; The method for automatically generating inspection points for the historical and cultural blocks to be inspected in step 1 is that the cloud computing platform automatically divides the historical and cultural blocks to be inspected into grids based on the input historical and cultural boundary data, and generates control points with a spacing of 10m along the roads based on the road data in the spatial database, and uses the control points as inspection points for the historical blocks.

3. The method for automatically profiling customers in historical and cultural blocks based on high-precision crowd flow monitoring according to claim 2 is characterized in that: In the step 2, the information of the inspection points in the historical and cultural blocks is automatically collected and uploaded, that is, the staff carries a backpack-type information collection device equipped with a WIFI probe, a 360° panoramic camera, a GPS positioning system and a human-computer interactive display screen, and a Moverio BT-40 model AR glasses, and reaches the designated marked inspection points through the human-computer interactive display screen for inspection. The stay time at each point is 1 minute. During the inspection, the staff clicks the start inspection button of this inspection point on the human-computer interactive display, and the backpack-type information collection device will start the information collection work for 1 minute. Through the crowd density monitoring unit and the scene information collection unit, combined with the WIFI probe, the 360° panoramic camera, and the GPS positioning system, the passenger flow density data, 360° panoramic photo data, time data, and geographic location data of the inspection points are automatically collected; After the information collection of the inspection point is completed, the backpack information collection device will automatically record the inspection status and inspection time of this inspection point, and upload the collected information data to the cloud computing platform through the collection information sending unit; wherein the crowd density monitoring unit is used to record the latitude and longitude information of all people collected by the backpack information collection device through the WIFI probe within a radius of 500m with the inspection point as the center when performing information collection work, and record the corresponding time, as well as the crowd density within a radius of 500m with the inspection point as the center, and store it as a file in the format of .xlsx according to the naming method of "collection time-collection inspection point"; the scene information collection unit is used to record the panoramic photo information of 1 photo taken per second by the 360° panoramic camera when the backpack information collection device performs 1 minute of information collection work at each inspection point, and record the corresponding time and longitude and latitude information, and store it as a .jpg file according to the naming method of "collection time-collection inspection point"; the collection information sending unit is used to send all information files collected by the crowd density monitoring unit and the scene information collection unit to the cloud computing platform; The method for automatically performing information verification on the cloud computing platform is that the cloud computing platform performs verification based on the information data collection standard. If the information verification fails, the cloud computing platform will feedback the inspection point where the information verification failed to the human-computer interaction display screen to guide the staff to perform secondary collection; wherein the information data collection standard refers to the occlusion rate of objects in the photos taken by the panoramic camera being less than 30%.

4. The method for automatically profiling customers in historical and cultural blocks based on high-precision crowd flow monitoring according to claim 3 is characterized in that: In the step 3, the passenger flow density data and the 360° panoramic photo data are preprocessed, that is, the customer group automatic profiling unit is built in a DGX computer workstation, and the 360° panoramic photo data is used to construct a crowd image data set, perform image grayscale processing, image filtering, edge detection and binarization processing, unify the image size and number it, and connect the time data and geographic location data with the crowd passenger flow density data and crowd image data; In the step three, a summary database of crowd information in the historical and cultural blocks is constructed and crowd ID numbering is performed, that is, the automatic customer portrait unit has a built-in DGX computer workstation, and the coordinate information of the crowd passenger flow density data set and the crowd image data set with coordinate information are unified into a unified spatial coordinate system, and the data information except the image information and coordinate information is converted into recognizable text information, and stored in the CSV file format after dimensionless processing; the crowd numbering is to obtain the scene image screen of each inspection point in the crowd image data set, extract and identify the face image in the scene image screen, and determine the temporary ID and location attributes of the face image; in the step three, the basic characteristics and spatiotemporal behavior characteristics of the mobile crowd in the historical and cultural blocks are identified, and the automatic customer portrait unit of the historical and cultural blocks is constructed based on the gender recognition module, age recognition module, attribute recognition module, emotion recognition module, and behavior recognition module of the built-in automatic customer portrait unit of the cloud computing platform to automatically portrait the historical and cultural blocks.

5. The method for automatically profiling customers in historical and cultural blocks based on high-precision crowd flow monitoring according to claim 4 is characterized in that: The method of automatically associating step 5 with the policy library in the cloud computing platform, that is, combining the customer group detection report output and the abnormal evaluation results of step 4, automatically associating different types of abnormal inspection points with the cloud computing platform policy library, and the associated information is the type of abnormal inspection point and the corresponding customer group passenger flow density value; The step 5 strategy library is combined with feedback information and real-time collected passenger flow data to convert them into visual information, that is, wearable smart devices are used to collect passenger flow data of the surrounding environment, and combined with GPS positioning, the analysis results of the cloud computing platform, real-time passenger flow data and related databases are fed back to the human-computer interaction display screen and Moverio BT-40 model AR glasses; the passenger flow data is combined with the data analysis results, converted into visual information in real time, and projected directly onto the display screen of the glasses, so that the staff can intuitively see the current passenger flow density, analysis results and corresponding strategies of each customer group; the system will automatically match the cloud computing analysis results, including different customer group portraits and their density in specific areas and abnormal customer groups. The area will be circled and highlighted in the augmented reality glasses with a pink icon, and the passenger flow density information of each relevant customer group and the relevant location map will be matched nearby with the real-time strategy corresponding to the strategy library.

Images