Traffic visualization monitoring system and method based on WIFI probe

Abstract

The invention relates to the technical field of monitoring, in particular to a traffic visualization monitoring system and method based on a WIFI probe, and the system comprises a data caching module, a system starting module, a configuration item reading module, a real-time processing module and a data display module, and uploads and processes data for a receiving probe; loading an environment quantity and an initialized detection point into a computing cluster; acquiring configuration attributes; acquiring real-time data, real-time traffic, real-time access area traffic and real-time access rate, and updating a database; and establishing WebSocket connection, acquiring data, initializing chart drawing and dynamically refreshing the data. Simulated data can be buffered to a server and then uploaded to a server cluster, data display is transmitted to the front of a front end through webstock, and a front-end page mainly applies an echrts and react framework technology to realize flow visualization, so that the problem that an existing monitoring system cannot realize flow visualization is solved.

Description

Technical Field

[0001] The present invention relates to the field of monitoring technology, and in particular to a flow visualization monitoring system and method based on WIFI probes. Background Art

[0002] In recent years, with the continuous development of smart phones, it has become more and more popular in our daily lives, and the WIFI component of mobile phones has gradually become an indispensable component of mobile phones. This article combines the WIFI function with attendance, and uses the WIFI probe method to determine whether people are within the specified range. The data obtained by the WIFI probe is used for efficient attendance, and the attendance data is displayed to people in real time in a visual large screen. The WIFI probe is based on wireless network detection technology, which can automatically identify mobile devices with WIFI detection function turned on within a certain range, including but not limited to smart phones, tablets, etc. It is a relatively popular hardware for detecting and obtaining the MAC address information of the device recently, similar to a router, and it can complete the search within the specified area by connecting to the power supply.

[0003] Traffic monitoring has a wide range of applications. For example, it can be applied to our daily attendance. Attendance is a common behavior in our lives. It is a way to monitor whether employees arrive at their workplaces during working hours. With the development of the times, the era of big data has arrived. Our attendance system should also keep pace with the times. It cannot be limited to fingerprint punching or facial recognition punching. Attendance should be combined with WIFI probes so that people can complete the punching-in related operations without actively operating. This improves work efficiency to a certain extent, and transmits people's attendance data to the front end in real time, and displays it in real time through a large visual screen, so that people can intuitively obtain attendance data. It is also of great significance to improve people's work efficiency. Summary of the invention

[0004] The purpose of the present invention is to provide a flow visualization monitoring system and method based on WIFI probe, aiming to solve the problem that the existing monitoring system cannot achieve flow visualization.

[0005] To achieve the above-mentioned purpose, in a first aspect, the present invention provides a flow visualization monitoring system based on a WIFI probe, comprising a data cache module, a system start module, a configuration item reading module, a real-time processing module and a data display module, wherein the data cache module, the system start module, the configuration item reading module, the real-time processing module and the data display module are connected in sequence;

[0006] The data cache module is used to upload and process data received by the probe;

[0007] The system start module is used to load environmental quantities and initialize detection points into the computing cluster;

[0008] The configuration item reading module is used to obtain configuration attributes;

[0009] The real-time processing module is used to obtain real-time data, real-time traffic, real-time entry area traffic, real-time entry rate and update the database;

[0010] The data display module is used to establish a WebSocket connection, obtain data, initialize chart drawing, and dynamically refresh data.

[0011] Wherein, the data cache module includes a data receiving unit, a data preprocessing unit, a data distribution unit and a data offline cache unit, the data receiving unit, the data preprocessing unit and the data distribution unit are connected in sequence, and the data offline cache unit is connected to the data preprocessing unit;

[0012] The data receiving unit is used to receive data uploaded by the probe;

[0013] The data preprocessing unit is used to re-encode the data, perform consistency analysis on the data, and complete missing fields;

[0014] The data distribution unit is used to distribute the processed data to the distributed message subscription system;

[0015] The data offline cache unit is used for offline caching of preprocessed data.

[0016] The system start module includes a Spark environment variable unit, a SQL environment variable unit, a Streaming environment variable unit, and an initialization checkpoint and data source unit;

[0017] The Spark environment variable unit is used to load SparkConf in the computing cluster;

[0018] The SQL environment variable unit is used to load SQLContext in the computing cluster;

[0019] The Streaming environment variable unit is used to load StreamingContext in the computing cluster;

[0020] The checkpoint initialization and data source unit is used to initialize the checkpoint directory and data source in the computing cluster.

[0021] Wherein, the real-time processing module includes a data acquisition unit, a real-time flow acquisition unit, a real-time entry area flow unit, a real-time entry rate unit and an update unit, and the data acquisition unit, the real-time flow acquisition unit, the real-time entry area flow unit, the real-time entry rate unit and the update unit are connected in sequence;

[0022] The data acquisition unit is used to acquire real-time data in the cluster;

[0023] The real-time traffic acquisition unit is used to analyze the total traffic of the real-time data;

[0024] The real-time inbound area flow unit is used to analyze the inbound area flow of the real-time data;

[0025] The real-time entry rate unit is used to analyze the entry rate of the real-time data;

[0026] The updating unit updates the database based on the total flow, the flow into the area and the entry rate.

[0027] Wherein, the real-time processing module comprises a connection unit, a query unit, a drawing unit and a refresh unit, and the connection unit, the query unit, the drawing unit and the refresh unit are connected in sequence;

[0028] The connection unit is used to establish a WebSocket connection with the backend;

[0029] The query unit is used to query the updated data in the database and push it to the front end for display;

[0030] The drawing unit is used by the front end to call the Echarts library to initialize the chart configuration items;

[0031] The refresh unit allows the user to refresh the chart.

[0032] In a second aspect, a traffic visualization monitoring method based on a WIFI probe is used in the traffic visualization monitoring system based on a WIFI probe as described in any one of the first aspects, comprising the following steps:

[0033] Upload data and process it to the receiving probe;

[0034] Load environmental quantities and initialize detection points into the computing cluster;

[0035] Get configuration properties;

[0036] Get real-time data, real-time traffic, real-time entry area traffic, real-time entry rate and update database;

[0037] Establish a WebSocket connection, obtain data, initialize chart drawing, and dynamically refresh data.

[0038] The present invention discloses a flow visualization monitoring system based on WIFI probe, comprising a data cache module, a system start module, a configuration item reading module, a real-time processing module and a data display module, wherein the data cache module, the system start module, the configuration item reading module, the real-time processing module and the data display module are connected in sequence; the data cache module is used to upload data and process the receiving probe; the system start module is used to load environmental quantities and initialize detection points into the computing cluster; the configuration item reading module is used to obtain configuration attributes; the real-time processing module is used to obtain real-time data, real-time flow, real-time flow into the area, real-time entry rate and update the database; the data display module is used to establish a WebSocket connection, obtain data, initialize chart drawing and dynamically refresh data. The simulated data of the present invention will be buffered to the server and then uploaded to the server cluster. The data display is transmitted to the front end through webstock. The front end page mainly uses echrts and react framework technologies to achieve flow visualization, thereby solving the problem that the existing monitoring system cannot achieve flow visualization. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The present invention is further illustrated by means of the non-limiting examples given in the accompanying drawings.

[0040] Figure 1 This is a schematic diagram of the user login interface

[0041] Figure 2 It is a schematic diagram of the real-time monitoring interface.

[0042] Figure 3 It is a schematic diagram of the user's real-time traffic interface.

[0043] Figure 4 This is a schematic diagram of the student ranking interface.

[0044] Figure 5 It is a flow analysis diagram.

[0045] Figure 6 It is a schematic diagram of the peak and average duration ratio interface.

[0046] Figure 7 It is a schematic diagram of a flow visualization monitoring system based on WIFI probe of the present invention.

[0047] Figure 8 is a schematic diagram of the data cache module.

[0048] Fig. 9 This is a schematic diagram of the system's starting module.

[0049] Fig.10is a schematic diagram of the real-time processing module.

[0050] Fig.11 It is a schematic diagram of the data display module.

[0051] Fig.12 The present invention is a flow chart of a method for visually monitoring traffic based on a WIFI probe.

[0052] In the figure: 1-data cache module, 2-system start module, 3-read configuration item module, 4-real-time processing module, 5-data display module, 11-data receiving unit, 12-data preprocessing unit, 13-data distribution unit, 14-data offline cache unit, 21-Spark environment variable unit, 22-SQL environment variable unit, 23-Streaming environment variable unit, 24-initialization checkpoint and data source unit, 41-data acquisition unit, 42-real-time traffic acquisition unit, 43-real-time entry area traffic unit, 44-real-time entry rate unit, 45-update unit, 51-connection unit, 52-query unit, 53-drawing unit, 54-refresh unit. DETAILED DESCRIPTION

[0053] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0054] See also Figure 1 To FIG. 11 , in a first aspect, the present invention provides a flow visualization monitoring system based on a WIFI probe, comprising a data cache module 1, a system start module 2, a configuration item reading module 3, a real-time processing module 4 and a data display module 5, wherein the data cache module 1, the system start module 2, the configuration item reading module 3, the real-time processing module 4 and the data display module 5 are connected in sequence;

[0055] The data cache module 1 is used to upload and process data received by the probe;

[0056] The system start module 2 is used to load environmental quantities and initialize detection points into the computing cluster;

[0057] The configuration item reading module 3 is used to obtain configuration attributes;

[0058] The real-time processing module 4 is used to obtain real-time data, real-time traffic, real-time entry area traffic, real-time entry rate and update the database;

[0059] The data display module 5 is used to establish a WebSocket connection, obtain data, initialize chart drawing, and dynamically refresh data.

[0060] In this embodiment, the data cache module 1 is used to upload data and process the receiving probe; the system start module 2 is used to load environmental quantities and initialize detection points into the computing cluster; the configuration item reading module 3 is used to obtain configuration attributes; the real-time processing module 4 is used to obtain real-time data, real-time traffic, real-time entry area traffic, real-time entry rate and update the database; the data display module 5 is used to establish a WebSocket connection, obtain data, initialize chart drawing and dynamically refresh data. The simulated data of the present invention will be buffered to the server and then uploaded to the server cluster. The data display is transmitted to the front end through webstock. The front-end page mainly uses echrts and react framework technology to achieve traffic visualization, thereby solving the problem that the existing monitoring system cannot achieve traffic visualization.

[0061] Further, the data cache module 1 includes a data receiving unit 11, a data preprocessing unit 12, a data distribution unit 13 and a data offline cache unit 14, the data receiving unit 11, the data preprocessing unit 12 and the data distribution unit 13 are connected in sequence, and the data offline cache unit 14 is connected to the data preprocessing unit 12;

[0062] The data receiving unit 11 is used to receive data uploaded by the probe;

[0063] The data preprocessing unit 12 is used for re-encoding the data, performing consistency analysis on the data and completing missing fields;

[0064] The data distribution unit 13 is used to distribute the processed data to the distributed message subscription system;

[0065] The data offline cache unit 14 is used for offline caching of pre-processed data.

[0066] In this embodiment, it is mainly used to receive data uploaded by the probe, preprocess the data, and push the data into the distributed message subscription system Kafka. It mainly has four steps: establishing a communication link between the probe and the server and obtaining data (this stage mainly includes data acquisition and data upload), data preprocessing function (this stage mainly includes the completion of missing fields and the deletion of redundant fields), data distribution (this stage mainly publishes data to the distributed subscription system Kafka) and offline data caching (this stage is mainly to obtain data for future machine learning preparation). These steps have the highest priority because data caching is the first level that the data obtained by the probe goes through, and the function of data caching module 1 is very important and indispensable.

[0067] Table 1 Functional description of data cache module 1

[0068]

[0069]

[0070]

[0071] Further, the system start module 2 includes a Spark environment variable unit 21, a SQL environment variable unit 22, a Streaming environment variable unit 23 and an initialization checkpoint and data source unit 24;

[0072] The Spark environment variable unit 21 is used to load SparkConf in the computing cluster;

[0073] The SQL environment variable unit 22 is used to load SQLContext in the computing cluster;

[0074] The Streaming environment variable unit 23 is used to load StreamingContext in the computing cluster;

[0075] The checkpoint initialization and data source unit 24 is used to initialize the checkpoint directory and data source in the computing cluster.

[0076] In this implementation, it is mainly divided into the following four functions: loading Spark environment variables, loading SQL environment variables, loading Streaming environment variables, and initializing checkpoints and data points. Each function is implemented in sequence according to the steps and is closely integrated to form the starting module of the system.

[0077] Table 2 Functional description of system start module 2

[0078]

[0079] Furthermore, the configuration item reading module 3 is used to obtain configuration attributes.

[0080] Table 3 Function description table of reading configuration item module 3

[0081]

[0082] In this embodiment, the configuration item reading module 3 is mainly divided into obtaining configuration attributes, and obtaining configuration attributes of string, int, long and boolean types, and these attribute contents are obtained from the configuration file through attribute conversion.

[0083] Further, the real-time processing module 4 includes a data acquisition unit 41, a real-time traffic acquisition unit 42, a real-time entry area traffic unit 43, a real-time entry rate unit 44 and an update unit 45, and the data acquisition unit 41, the real-time traffic acquisition unit 42, the real-time entry area traffic unit 43, the real-time entry rate unit 44 and the update unit 45 are connected in sequence;

[0084] The data acquisition unit 41 is used to acquire real-time data in the cluster;

[0085] The real-time traffic acquisition unit 42 is used to analyze the total traffic of the real-time data;

[0086] The real-time inbound area flow unit 43 is used to analyze the inbound area flow of the real-time data;

[0087] The real-time entry rate unit 44 is used to analyze the entry rate of the real-time data;

[0088] The updating unit 45 updates the database based on the total flow, the flow into the area and the entry rate.

[0089] In this embodiment, data is obtained, real-time traffic is obtained, real-time entry area traffic is obtained, real-time entry rate is obtained, and database is updated, and each function is performed sequentially according to steps.

[0090] Table 4 Functional description of real-time processing module 4

[0091] serial number name describe Priority 001 Get data Reading real-time data from a Kafka cluster high 002 Get real-time traffic Analyze real-time data in the cluster to get the total traffic high 003 Get real-time inbound zone traffic Analyze real-time data in the cluster to obtain the traffic entering the region high 004 Get real-time access rate Analyze real-time data in the cluster to get the entry rate high 005 Update the database Write the analysis results to the database for front-end display high

[0092] Further, the real-time processing module 4 includes a connection unit 51, a query unit 52, a drawing unit 53 and a refresh unit 54, and the connection unit 51, the query unit 52, the drawing unit 53 and the refresh unit 54 are connected in sequence;

[0093] The connection unit 51 is used to establish a WebSocket connection with the backend;

[0094] The query unit 52 is used to query the updated data in the database and push it to the front end for display;

[0095] The drawing unit 53 is used for the front end to call the Echarts library to initialize the chart configuration items;

[0096] The refresh unit 54 allows the user to refresh the chart.

[0097] In this embodiment, a WebSocket connection is established, data is acquired, chart drawing is initialized, and data is dynamically refreshed.

[0098] Table 5 Functional description of real-time processing module 4

[0099]

[0100] Please participate Fig.12 In a second aspect, a method for visually monitoring traffic based on a WIFI probe is provided, which is used in the visually monitoring traffic based on a WIFI probe as described in any one of the first aspects, and comprises the following steps:

[0101] S1 uploads data and processes the receiving probe;

[0102] Specifically, the data receiving unit 11 is used to receive data uploaded by the probe; the data preprocessing unit 12 is used to re-encode the data and perform consistency analysis on the data and complete missing fields; the data distribution unit 13 is used to distribute the processed data to a distributed message subscription system; the data offline caching unit 14 is used to perform offline caching of the preprocessed data.

[0103] S2 loads environmental quantities and initialization detection points into the computing cluster;

[0104] Specifically, the Spark environment variable unit 21 is used to load SparkConf in the computing cluster; the SQL environment variable unit 22 is used to load SQLContext in the computing cluster; the Streaming environment variable unit 23 is used to load StreamingContext in the computing cluster; the initialization checkpoint and data source unit 24 is used to initialize the checkpoint directory and data source in the computing cluster.

[0105] S3 obtains configuration attributes;

[0106] Specifically, configuration attributes are obtained, as well as configuration attributes of string, int, long, and boolean types. These attribute contents are obtained from the configuration file through attribute conversion.

[0107] S4 obtains real-time data, real-time traffic, real-time entry area traffic, real-time entry rate and updates the database;

[0108] Specifically, the data acquisition unit 41 is used to acquire real-time data in the cluster; the real-time traffic acquisition unit 42 is used to analyze the total traffic of the real-time data; the real-time entry area traffic unit 43 is used to analyze the entry area traffic of the real-time data; the real-time entry rate unit 44 is used to analyze the entry rate of the real-time data; the update unit 45 updates the database based on the total traffic, the entry area traffic and the entry rate.

[0109] S5 establishes a WebSocket connection, obtains data, initializes chart drawing, and dynamically refreshes data.

[0110] Specifically, the connection unit 51 is used to establish a WebSocket connection with the backend; the query unit 52 is used to query the updated data in the database and push it to the frontend for display; the drawing unit 53 is used for the frontend to call the Echarts library to initialize the chart configuration items; the refresh unit 54 is used for the user to refresh the chart.

[0111] What is disclosed above is only one or more preferred embodiments of the present application, and cannot be used to limit the scope of rights of the present application. Ordinary technicians in this field can understand that all or part of the processes of implementing the above embodiments and equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A traffic visualization monitoring system based on WIFI probe, characterized by: It includes a data cache module, a system start module, a configuration item reading module, a real-time processing module and a data display module, wherein the data cache module, the system start module, the configuration item reading module, the real-time processing module and the data display module are connected in sequence; The data cache module is used to upload and process data received by the probe; The system start module is used to load environmental quantities and initialize detection points into the computing cluster; The configuration item reading module is used to obtain configuration attributes; The real-time processing module is used to obtain real-time data, real-time traffic, real-time entry area traffic, real-time entry rate and update the database; The data display module is used to establish a WebSocket connection, obtain data, initialize chart drawing, and dynamically refresh data.

2. The flow visualization monitoring system based on WIFI probe as claimed in claim 1, characterized in that: The data cache module includes a data receiving unit, a data preprocessing unit, a data distribution unit and a data offline cache unit, wherein the data receiving unit, the data preprocessing unit and the data distribution unit are connected in sequence, and the data offline cache unit is connected to the data preprocessing unit; The data receiving unit is used to receive data uploaded by the probe; The data preprocessing unit is used to re-encode the data, perform consistency analysis on the data, and complete missing fields; The data distribution unit is used to distribute the processed data to the distributed message subscription system; The data offline cache unit is used for offline caching of preprocessed data.

3. The flow visualization monitoring system based on WIFI probe as claimed in claim 1, characterized in that: The system start module includes a Spark environment variable unit, a SQL environment variable unit, a Streaming environment variable unit, and an initialization checkpoint and data source unit; The Spark environment variable unit is used to load SparkConf in the computing cluster; The SQL environment variable unit is used to load SQLContext in the computing cluster; The Streaming environment variable unit is used to load StreamingContext in the computing cluster; The checkpoint initialization and data source unit is used to initialize the checkpoint directory and data source in the computing cluster.

4. The flow visualization monitoring system based on WIFI probe as claimed in claim 1, characterized in that: The real-time processing module includes a data acquisition unit, a real-time traffic acquisition unit, a real-time entry area traffic unit, a real-time entry rate unit and an update unit, wherein the data acquisition unit, the real-time traffic acquisition unit, the real-time entry area traffic unit, the real-time entry rate unit and the update unit are connected in sequence; The data acquisition unit is used to acquire real-time data in the cluster; The real-time traffic acquisition unit is used to analyze the total traffic of the real-time data; The real-time inbound area flow unit is used to analyze the inbound area flow of the real-time data; The real-time entry rate unit is used to analyze the entry rate of the real-time data; The updating unit updates the database based on the total flow, the flow into the area and the entry rate.

5. The flow visualization monitoring system based on WIFI probe as claimed in claim 1, characterized in that: The real-time processing module includes a connection unit, a query unit, a drawing unit and a refresh unit, wherein the connection unit, the query unit, the drawing unit and the refresh unit are connected in sequence; The connection unit is used to establish a WebSocket connection with the backend; The query unit is used to query the updated data in the database and push it to the front end for display; The drawing unit is used by the front end to call the Echarts library to initialize the chart configuration items; The refresh unit allows the user to refresh the chart.

6. A method for visually monitoring traffic based on a WIFI probe, used in the visually monitoring traffic based on a WIFI probe according to any one of claims 1 to 5, characterized in that: The following steps are involved: Upload data and process it to the receiving probe; Load environmental quantities and initialize detection points into the computing cluster; Get configuration properties; Get real-time data, real-time traffic, real-time entry area traffic, real-time entry rate and update database; Establish a WebSocket connection, obtain data, initialize chart drawing, and dynamically refresh data.

Images