A highway data receiving, processing and storing method and system

Abstract

A highway data receiving processing storage method and system, comprising the following steps: receiving monitoring data of the highway; classifying and extracting the monitoring data according to a classification rule to obtain extraction data; setting the extraction data into a corresponding classification database to obtain storage data; and the classification rule is established according to the data format of the classification database. The application is based on the background of realizing nationwide network by canceling provincial boundary toll stations. Due to the fact that there are many manufacturers and different protocol standards in various toll gantries, toll lanes and overspeed control systems in the highway, corresponding protocol development for manufacturers is often required, resulting in a large amount of repetitive work. In order to reduce repetitive work, the classification rule is established according to the data format of the classification database, and the classification of the monitoring data is based on this.

Description

TECHNICAL FIELD

[0001] The application relates to a highway data receiving processing and storing method and system. BACKGROUND

[0002] Under the background of canceling the provincial boundary toll station to realize the nationwide network, there are various toll gantries, toll lanes and super system in the highway, and there are many manufacturers and different protocol standards, and corresponding protocol development is often needed for manufacturers, resulting in a large amount of repetitive work. Now there are some ways of storing through data analysis, but the overall efficiency is low, and errors are prone to occur in the conversion process, and it is not suitable for use in the field of highway data which has high requirements. SUMMARY

[0003] In order to solve the above problems, the application discloses a highway data receiving processing and storing method, which comprises the following steps:

[0004] receiving monitoring data of the highway;

[0005] classifying and extracting the monitoring data according to a classification rule to obtain extracted data;

[0006] setting the extracted data into a corresponding classification database to obtain stored data;

[0007] The classification rule is established according to the data format of the classification database. The application is based on the background of canceling the provincial boundary toll station to realize the nationwide network. Since there are various toll gantries, toll lanes and super system in the highway, and there are many manufacturers and different protocol standards, corresponding protocol development is often needed for manufacturers, resulting in a large amount of repetitive work. In order to reduce the repetitive work content, the classification rule is established according to the data format of the classification database, and the classification of the monitoring data is based on this.

[0008] Preferably, the process of creating the classification database and the classification rule comprises the following steps: new establishment of the classification database according to the monitoring data, and determination of the classification rule according to the data structure of the classification database.

[0009] Preferably, the classification extraction is comprehensively judged according to the data source and the data self-identification analysis.

[0010] Preferably, the comprehensive judgment is performed in the following manner: the data source is obtained and confirmed according to the original source provided by the monitoring data, and the pre-classification of the monitoring data is performed according to the historical information of the monitoring data provided by the original source.

[0011] The target data is matched with the data in the classification database of the pre-classified type through extraction of the target data in the monitoring data, and if the matching rate of the data matching is lower than a threshold, the target data is matched with the data in other classification databases until the matching rate of the data matching is not lower than the threshold;

[0012] If a classification database meeting the requirement is not obtained in the process, and if the matching rate of at least one of the classification databases exceeds 60%, the extracted data is written into the classification database with the highest matching rate, and the processed target data is marked as abnormal data. The comprehensive evaluation of the application can ensure the reduction of the required time for data discrimination and the accuracy of classification of the monitoring data, and obtain the abnormal data, thereby providing a basis for effective expansion of the classification database.

[0013] Preferably, the data matching includes numerical value matching and data representation form matching.

[0014] Preferably, the method further comprises analysis and monitoring of specific data in the stored data: a median value of the stored data is calculated, and data within a range of 30% above and below the median value is confirmed as normal data; the normal data at the boundary is used as a base point, and outward expansion calculation is performed, if the numerical value change rate on both sides is lower than 10% of the normal data used as the calculation base point, the adjacent stored data is marked as normal data, and the process is repeated until more than 10% of the stored data is found or all the calculation is completed, and the more than 10% of the stored data is marked as specific data; the specific data is subjected to source verification and data extraction accuracy verification. The application can determine the data extraction accuracy through self-determination of the internal data, so as to avoid the addition and accumulation of specific data, and improve the accuracy of the stored data itself.

[0015] Preferably, the method further comprises a processing process of the abnormal data: when the data amount of the abnormal data in the same classification database exceeds a data amount threshold or the percentage exceeds a percentage threshold, a new classification database is established through the data source and the meaning of the abnormal data.

[0016] Preferably, the monitoring data includes lane monitoring data, gantry monitoring data and overload control monitoring data.

[0017] Preferably, the database is a JSON database.

[0018] In another aspect, the application further discloses a highway data receiving, processing and storing system, comprising the following modules:

[0019] The data receiving module is configured to receive monitoring data of the highway.

[0020] A data extraction module is configured to extract the monitoring data according to the classification rule to obtain extraction data.

[0021] A data classification module is configured to set the extraction data into a corresponding classification database.

[0022] A configuration module is configured to set the classification database and the classification rule.

[0023] The present application can bring the following beneficial effects:

[0024] 1. The present application is based on the background of canceling the provincial boundary toll station to realize the nationwide network. Due to the fact that there are many manufacturers and different protocol standards in various types of toll gantries, toll lanes and overspeed control systems in the expressway, it is often necessary to develop the corresponding protocol for the manufacturers, which leads to a large amount of repetitive work. In order to reduce the repetitive work, the classification rule is established according to the data format of the classification database, and the classification of the monitoring data is performed based on this.

[0025] 2. The comprehensive judgment of the present application can judge the data type through the data source, which can ensure the reduction of the required time when judging the data and the accuracy of the classification of the monitoring data, and obtain the special data, which provides a basis for the effective expansion of the classification database.

[0026] 3. The present application can judge the accuracy of data extraction through self-judgment of internal data, so as to avoid the addition and accumulation of special data and improve the accuracy of the storage data itself. BRIEF DESCRIPTION OF DRAWINGS

[0027] The drawings described herein are used to provide further understanding of the present application, and form a part of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0028] Figure 1 is a schematic diagram of embodiment 1;

[0029] Figure 2 is a schematic diagram of embodiment 2. DETAILED DESCRIPTION

[0030] In order to clearly illustrate the technical features of the present application, the present application will be described in detail below through specific embodiments.

[0031] In the first embodiment, as shown in Figure 1 , a highway data receiving processing and storage method includes the following steps:

[0032] S101. Receiving monitoring data of the expressway;

[0033] The monitoring data includes lane monitoring data, gantry monitoring data and overload control monitoring data.

[0034] S102. The monitoring data is classified and extracted according to the classification rules, which are established according to the data format of the classification database.

[0035] The process of creating the classification database and the classification rules includes the following steps:

[0036] The classification database is newly created according to the monitoring data, and then the classification rules are determined according to the data structure of the classification database.

[0037] The classification extraction is comprehensively evaluated according to the data source and the data itself.

[0038] The comprehensive evaluation is performed in the following way:

[0039] The data source is confirmed according to the original source provided by the monitoring data, and the pre-classification of the monitoring data is performed according to the historical information of the monitoring data provided by the original source. For example, if the historical information shows that it is mostly lane monitoring data, then the classification is performed according to the lane monitoring data.

[0040] The data matching is performed between the target data in the monitoring data and the data in the classification database of the pre-classified type. If the matching rate of the data matching is lower than the threshold value, such as 80%, then the data matching is performed between the target data and the data in other classification databases, until the matching rate of the data matching is not lower than the threshold value.

[0041] If a classification database that meets the requirements is not obtained in this process, and if the matching rate of at least one of them exceeds 60%, then the extracted data is written into the classification database with the highest matching rate, and the processed data is marked as abnormal data.

[0042] The data matching includes numerical matching and data representation matching. If there are the same numerical values or numerical values within 5%, then the matching rate can be defined as 100%. If there is no data within this range or more than 5%, then the matching rate is set to 90%, 85%, etc. according to the degree of exceeding.

[0043] The processing process of the abnormal data:

[0044] When the data amount of the abnormal data in the same classification database exceeds the data amount threshold (such as more than 10,000) or the percentage exceeds the percentage threshold (such as more than 20%, i.e. the total data is 10,000 and the abnormal data is 2,000), then a new classification database is established according to the data source and the meaning of the abnormal data.

[0045] S103. The extracted data is set into the corresponding classification database to obtain stored data;

[0046] Analysis monitoring of specific data in the stored data:

[0047] The median of the stored data is calculated, and data within 30% of the median is confirmed as normal data;

[0048] The normal data at the boundary is used as a base point for outward expansion calculation. If the rate of increase and decrease of the values on both sides is lower than 10% of the normal data as the calculation base point, the adjacent stored data is marked as normal data. This is repeated until more than 10% of the stored data is found or all calculations are completed. More than 10% of the stored data is marked as specific data.

[0049] If the value of the normal data at the boundary is 100, the nearest normal data is 101, and the data to be verified is 99, the calculation is: [(100-99)-(101-99)] / 100=0, then it is marked as normal data, and the calculation is repeated.

[0050] The source verification (i.e. original source) and data extraction accuracy verification are performed for the specific data (for data accuracy, manual verification can be performed, or data re-extraction can be performed through re-extraction).

[0051] It can be understood that the database is a JSON database.

[0052] In the second embodiment, as shown in Figure 2 A highway data receiving processing and storage system includes the following modules:

[0053] The data receiving module 201 is used to receive monitoring data of the highway.

[0054] The data extraction module 202 is used to classify and extract the monitoring data to obtain extracted data according to classification rules.

[0055] The data classification module 203 is used to set the extracted data into the corresponding classification database.

[0056] The configuration module 204 is used to set the classification database and the classification rules.

[0057] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various changes and variations. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application shall be included in the scope of the claims of the present application.

Claims

1. A method for receiving, processing, and storing highway data, characterized in that: Includes the following steps: Receive monitoring data from highways; The monitoring data is classified and extracted according to the classification rules to obtain the extracted data; Extracted data is set into the corresponding category database to obtain stored data; The classification rules are established according to the data format of the classification database; This also includes the analysis and monitoring of anomalies within the stored data: Calculate the median of the stored data and confirm that data within 30% above and below the median are ordinary data. Using ordinary data at the boundary as the base point, the calculation is expanded outward. If the rate of increase or decrease of the values ​​on both sides is less than 10% of the calculation base point using ordinary data, then the adjacent stored data is marked as ordinary data. This process is repeated until more than 10% of the stored data is found or all calculations are completed. More than 10% of the stored data is marked as special data. For special data, source verification and accuracy verification of data extraction are required.

2. The method for receiving, processing, and storing highway data according to claim 1, characterized in that: It also includes the process of creating a classification database and classification rules, which includes the following steps: A new classification database is created based on the monitoring data, and then the classification rules are determined based on the data structure of the classification database.

3. The method for receiving, processing, and storing highway data according to claim 1, characterized in that: The classification and extraction are obtained by comprehensively evaluating the data source and the data itself through identification and analysis.

4. The method for receiving, processing, and storing highway data according to claim 3, characterized in that: The comprehensive evaluation is conducted in the following manner: The data source is obtained and confirmed according to the original source provided by the monitoring data, and the monitoring data is pre-classified based on the historical information of the monitoring data provided by the original source; By extracting target data from the monitoring data, the target data is matched with data in the pre-classified type classification database. If the matching rate is lower than the threshold, the target data is matched with data in other classification databases until the matching rate is not lower than the threshold. If no suitable classification database is obtained during this process, and if at least one of them has a matching rate of over 60%, the extracted data will be written into the classification database with the highest matching rate and marked to obtain the outlier data.

5. The method for receiving, processing, and storing highway data according to claim 4, characterized in that: The data matching includes numerical matching and matching of data representation formats.

6. The method for receiving, processing, and storing highway data according to claim 4, characterized in that: It also includes the processing of outlier data: When the amount of outlier data in the same classification database exceeds the data volume threshold or the percentage exceeds the percentage threshold, a new classification database is established based on the data source and meaning of the outlier data.

7. The method for receiving, processing, and storing highway data according to claim 1, characterized in that: The monitoring data includes lane monitoring data, gantry monitoring data, and overload control monitoring data.

8. The method for receiving, processing, and storing highway data according to claim 1, characterized in that: The database is a JSON database.

9. A highway data receiving, processing, and storage system for implementing the highway data receiving, processing, and storage method according to any one of claims 1-8, characterized in that: Includes the following modules: The data receiving module is used to receive monitoring data from the highway. The data extraction module is used to classify and extract monitoring data according to classification rules to obtain the extracted data; The data classification module is used to set the extracted data into the corresponding classification database; The configuration module is used to set the classification database and classification rules.

Images